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LONGITUDINAL  VALIDATION  OF  NON-COGNITIVE  OFFICER  SELECTION 
MEASURES  FOR  THE  U.S.  ARMY  OFFICER  CANDIDATE  SCHOOL  (OCS) 

EXECUTIVE  SUMMARY 


Research  Requirement: 

Background.  The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  has  a  broad,  ongoing  effort  to  identify,  develop,  and  validate  personnel  tests  for  use  in 
officer  selection  and  branch  assignments.  The  primary  goal  of  this  program  is  to  recommend  and 
implement  such  measures  that  will  support  efforts  to  enhance  both  officer  performance  and 
career  continuance  across  commissioning  sources.  The  effort  described  in  this  report  focuses 
specifically  on  procedures  for  selecting  candidates  for  the  U.S.  Army’s  Officer  Candidate 
School. 


Officer  Candidate  School  (OCS)  is  an  important  commissioning  source  for  Army 
officers.  Historically,  the  Army  has  used  OCS  to  fill  accession  gaps  in  officer  requirements  when 
other  commissioning  sources  were  unable  to  meet  force  structure  needs.  Compared  to  the 
Reserve  Officer  Training  Corps  (ROTC)  and  the  U.S.  Military  Academy  (USMA),  OCS  is 
flexible  enough  to  increase  or  decrease  its  rate  of  commissioning  officers  on  short  notice. 

First  Phase  of  OCS  Research.  The  first  phase  of  ARI’s  OCS  research  (Russell  & 

Tremble,  2011)  served  as  the  foundation  for  the  effort  described  in  this  report.  The  first  phase 
began  in  2008,  when  researchers  administered  an  extensive  battery  of  personnel  selection  tests  to 
over  1,300  entering  OCS  candidates.  This  test  battery,  the  Officer  Background  and  Experiences 
Form  (OBEF),  included  measures  of  temperament,  affectivity,  values,  and  leadership.  One  of 
these  measures  was  a  version  of  the  Rational  Biodata  Inventory  (RBI).  The  RBI  is  a  non- 
cognitive  measure  of  personality  and  motivational  attributes  that  has  shown  promise  for 
enhancing  selection  of  cadets  in  ROTC  (Putka,  Wasko,  Tremble,  Kilcullen,  &  Shaw,  2009; 
Wasko,  Putka,  Legree,  &  Kilcullen,  2011)  and  enlisted  Soldiers  (Knapp  &  Heffner,  2009; 

Knapp,  Owens,  &  Allen,  2011).  Instruments  measuring  work  values  and  judgment  in  leadership 
situations  also  were  administered  as  part  of  the  OBEF.  We  used  a  longitudinal  validation 
research  design  to  track  these  tested  candidates  through  the  end  of  their  12-week  OCS  training 
course.  In  addition,  approximately  50%  of  these  individuals  were  further  tracked  into  their  Basic 
Officer  Leadership  Course  III,  where  newly  commissioned  officers  complete  their  branch- 
specific  training.  Data  collected  both  at  the  end  of  OCS  and  the  period  during  BOLC  III 
(approximately  6  months  following  OCS  graduation)  were  used  to  obtain  early  indicators  of 
perfonnance  and  continuance  (e.g.,  OCS  training  performance  scores,  self-reported  commitment, 
and  career  continuance  intentions)  for  the  longitudinal  validation. 

The  preliminary  findings  from  this  first  phase  of  OCS  research  showed  that  a  subset  of 
the  OBEF  was  predictive  of  practical  and  conceptually  important  outcomes  such  as  commitment 
to  the  Anny,  career  continuance  intentions,  and  OCS  perfonnance  scores.  Importantly,  these 
tests  demonstrated  incremental  validity,  beyond  that  of  the  Anned  Services  Vocational  Aptitude 
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Battery  (ASVAB) — which  the  Army  uses  for  screening  applicants  into  OCS.  In  addition,  the 
findings  were  similar  across  both  the  enlistment  option  (for  civilians  selected  directly  into  OCS) 
and  in-service  (for  enlisted  Soldiers  entering  OCS)  accession  programs. 

Current  Research  Objectives.  The  specific  objectives  of  the  current  effort  were  to  (a) 
collect  longer-tenn  criterion  measures  (e.g.,  in-unit  performance,  service  continuance)  for  the 
members  of  the  2008  OCS  cohort,  (b)  validate  the  2008  test  battery  (the  OBEF)  against  these 
longer-term  outcomes,  (c)  validate  the  OCS  selection  composites  developed  in  the  first  phase 
against  the  same  longer-tenn  outcomes,  and  (d)  model  changes  in  officer  attitudes  during  their 
initial  Active  Duty  Service  Obligation,  and  the  antecedents  and  consequences  of  those  changes. 

These  objectives  were  driven  in  large  part  by  a  need  to  transition  ARI’s  experimental 
OCS  selection  tests  to  operational  use  in  an  Initial  Operational  Test  and  Evaluation  (IOT&E) 
within  the  next  few  years.  The  current  research  was  designed  to  help  inform  decisions  regarding 
the  choice  of  the  operational  test  battery  and  the  Army’s  officer  candidate  selection  policies. 

Procedure: 

At  the  beginning  of  this  project  in  early  2011,  the  junior  officers  in  the  2008  OCS  cohort 
were  approaching  the  end  of  their  3-year  Active  Duty  Service  Obligation  (ADSO).  This  was  a 
critical  period  as  these  officers  had  gained  substantial  experience  working  in  their  initial  units 
and  were  deciding  whether  or  not  to  continue  serving  in  the  Regular  Army.  We  developed  data 
collection  approaches  and  measures  that  captured  important  career  outcome  infonnation  for 
these  young  officers  during  this  key  period. 

We  developed  and  fielded  a  web-based  “In-Unit  Survey”  for  measuring  the  officers’ 
attitudes,  performance,  and  continuance  intentions.  The  perfonnance-related  items  in  the  survey 
covered  topics  such  as  awards  received,  ratings  obtained  on  their  most  recent  Officer  Evaluation 
Report  (OER),  scores  on  the  Army  Physical  Fitness  Test  (APFT),  weapon’s  qualification,  and 
self-assessments  on  dimensions  of  in-unit  performance  (e.g.,  Technical  Competence).  The  survey 
also  included  items  addressing  the  officers’  active  duty  career  intentions  and  whether  or  not  they 
had  (or  planned  to)  submit  paperwork  requesting  separation  from  the  Regular  Anny.  We 
constructed  additional  outcome  measures  (e.g.,  graduation  from  officer  training,  number  of 
medals  received)  using  data  extracted  from  administrative  personnel  and  training  databases. 

A  longitudinal  validation  database  was  constructed  by  integrating  the  survey  and 
administrative  data  collected  under  this  effort  with  the  existing  OCS  cohort  data.  This  integrated 
database  included  (a)  the  OBEF  instrument  data  (predictors)  collected  at  the  beginning  of  OCS  in 
2008,  (b)  attitudinal  and  perfonnance  measures  collected  at  the  completion  of  OCS,  (c) 
attitudinal  and  perfonnance  measures  captured  during  the  Basic  Officer  Leadership  Course  III 
(BOLC  III;  approximately  6  months  after  graduation  from  OCS),  and  (d)  the  In-Unit  Survey  and 
administrative  data  collected  under  the  current  effort. 

Expert  input  guided  the  development  of  conceptually-based  measures  of  criterion 
dimensions  for  use  in  analyses.  We  constructed  these  measures  from  items  in  the  In-Unit  Survey 
as  well  as  data  in  the  automated  personnel  and  training  databases.  The  constructed  criterion 
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dimensions  included  Branch-Specific  Technical  Proficiency,  Army-Wide  Technical  Proficiency, 
Personal  Discipline,  Physical  Fitness,  Leadership,  Career  Continuance  Intentions,  Branch 
Satisfaction,  Commitment  to  the  Army,  and  Morale.  We  also  examined  in-unit  records  of  officer 
performance  (i.e.,  number  of  awards  received)  and  separations  (e.g.,  separation  during  training, 
separation  at  the  end  of  their  ADSO)  as  additional  outcome  criteria. 

Findings: 

Predictive  Validation  of  the  OBEF.  Although  the  OBEF  validation  findings  varied  by 
subtest/criterion/subgroup  combination,  the  overall  pattern  of  results  clearly  showed  that  the 
OBEF  was  predictive  of  key  officer  performance  and  continuance  outcomes  approximately  three 
years  after  their  time  of  commissioning.  Specifically,  aspects  of  the  OBEF  predicted  the 
occurrence  of  training  recycles  (i.e.,  failing  or  having  to  repeat  some  aspect  of  an  officer  training 
program),  commitment  to  the  Anny,  physical  fitness,  technical  knowledge,  and  career 
continuance  intentions  for  officers  that  entered  OCS  through  both  the  enlistment  option  and  in- 
service  accession  programs.  However,  certain  scales  predicted  key  outcomes  in  one  sample,  and 
not  the  other.  For  example,  valuing  “skill  development”  in  one’s  work  was  predictive  of  multiple 
outcomes  in  the  in-service  sample,  but  not  in  the  enlistment  option  sample.  In  predicting 
perfonnance  and  continuance  outcomes,  the  OBEF  demonstrated  incremental  validity  beyond 
that  of  the  Armed  Forces  Qualification  Test  (AFQT),  a  key  part  of  the  ASVAB.  This  finding  is 
critical,  given  that  the  ASVAB  is  used  as  a  major  component  in  the  Army’s  current  officer 
selection  system. 

Of  particular  note,  the  OBEF  composites  developed  in  2008  for  selecting  in-service  and 
enlistment  option  officers  into  OCS  predicted  key  perfonnance  and  continuance  outcomes  three 
years  later.  Specifically,  the  in-service  composite  predicted  key  attitudinal  (commitment,  branch 
satisfaction,  morale)  and  performance  (physical  fitness)  outcomes,  while  the  enlistment  option 
composite  predicted  commitment,  physical  fitness,  completion  of  initial  officer  training,  and 
active  duty  separation. 

Modeling  Changes  in  Officer  Attitudes  Over  Time.  A  secondary  purpose  of  this  research 
effort  was  to  examine  changes  in  officer  attitudes  over  time,  the  factors  that  predict  these  attitude 
changes,  and  the  consequences  of  negative  changes  in  attitudes  over  their  initial  ADSO.  Results 
from  the  Latent  Growth  Modeling  (LGM)  showed  that  officers  entering  OCS  through  the  in- 
service  and  enlistment  option  programs  differed  in  their  change  trajectories  for  multiple 
attitudinal  outcomes.  For  example,  commitment  generally  decreased  over  time  for  in-service 
sample,  but  stayed  relatively  stable  for  the  enlistment  option  sample.  In  general,  most  attitudes 
exhibited  either  no  change  or  negative  change  trajectories  over  time.  In  modeling  predictors  and 
outcomes  of  these  attitude  changes,  we  found  that  changes  in  morale  could  be  predicted  by  the 
OBEF,  and  above  average  change  in  morale  was  associated  with  staying  on  active  duty  service 
after  the  end  of  their  ADSO.  Finally,  we  found  that  self-reported  career  continuance  intentions 
were  positively  associated  with  actual  active  duty  service  continuance  (vs.  separation)  behavior. 
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Utilization  and  Dissemination  of  Findings: 

The  research  findings  reported  here  provided  additional  evidence  supporting  the  use  of 
the  Officer  Background  and  Experiences  Form  (OBEF)  as  an  OCS  selection  tool.  Moreover, 
these  findings  infonn  the  refinement  of  the  OBEF  as  the  Army  moves  it  toward  operational  use. 
An  Initial  Operational  Test  and  Evaluation  (IOT&E)  of  the  OBEF  for  enlistment  option 
applicants  currently  is  being  planned  as  an  interim  step  in  this  process.  Some  of  the  new 
measures  and  data  collection  methodologies  developed  under  this  effort  will  likely  be  used  to 
support  other  related  ARI  officer  selection  research  projects  in  the  future.  Findings  from  this 
research  have  been  reported  to  the  OCS  Commandant  (April  2012),  the  Chief,  Enlisted 
Accession  Division,  Army-G-1  (June  2012),  and  the  Chief,  Officer  Division,  Army  G-l  (June 
2012). 


viii 


LONGITUDINAL  VALIDATION  OF  NON-COGNITIVE  OFFICER  SELECTION 

MEASURES  FOR  THE  U.S.  ARMY  OFFICER  CANDIDATE  SCHOOL  (OCS) 

CONTENTS _ 

Page 

Chapter  1 :  Introduction . 1 

Background . 1 

Officer  Candidate  School:  Background  and  Context . 1 

Preliminary  OCS  Research  Effort . 2 

OCS  Longitudinal  Validation  Research . 3 

Objectives . 3 

Challenges . 4 

Organization  of  Report . 4 

Chapter  2:  Research  Methodology . 6 

Research  Overview . 6 

Officer  Background  and  Experiences  Fonn  (OBEF) . 9 

In-Unit  (Time  4)  Survey  Development . 10 

Data  Collection  Procedures . 13 

In-Unit  Survey  Administration . 13 

Administrative  Data  Collections . 15 

Ongoing  Data  Collections . 18 

In-Unit  Survey  Scoring . 19 

Summary . 21 

Chapter  3:  Prediction  of  Officer  Performance  and  Continuance  with  the  OBEF . 25 

Background . 25 

Analytic  Approach . 26 

Missing  Data  Estimation . 27 

Incremental  Validity . 27 

Range  Restriction . 28 

Shrinkage . 29 

Incremental  Validity  Results . 30 

Enlistment  Option  Results . 30 

In-Service  Results . 34 

Conclusion . 38 

Chapter  4:  Longitudinal  Examination  of  Officer  Attitudes  Over  Time . 40 

Approach . 40 

Results . 44 

Change  Over  Time . 44 

Differences  in  Changes  over  Time  by  Entry  Status . 48 

Predictors  and  Outcomes  of  Change . 49 

ix 


CONTENTS  (continued) 


Page 

Discussion . 50 

Longitudinal  Change . 50 

Group  Differences . 51 

Predictors  and  Outcomes . 51 

Conclusion . 51 

Chapter  5:  Conclusions  and  Next  Steps . 52 

Summary  of  Methodological  Approach . 52 

Summary  of  Results . 54 

Limitations . 56 

Future  Directions . 57 

Longitudinal  Database  Development  and  Validation . 57 

The  Efficacy  of  the  OBEF  for  Operational  Use . 58 

REFERENCES . 59 

Appendix  A  Core  RBI  and  Work  Values  Component  Scale  Descriptions . A-l 

Appendix  B  In-Unit  (Time  4)  Survey  Content . B-l 

Appendix  C  Subsequent  Data  Collections  and  Instruments . C-l 

Appendix  D  Bivariate  Correlation  Tables . D-l 

LIST  OF  TABLES 

Table  2.1.  Officer  Candidate  School  (OCS)  Sample  Demographics  for  In-Unit  Survey 

Respondents . 14 

Table  2.2.  Officer  Candidate  School  (OCS)  Full  Sample  Demographics . 15 

Table  2.3.  Summary  of  Potential  Criterion  Variables  Created  Using  ATRRS . 16 

Table  2.4.  Valor  and  Merit  Awards  with  Associated  Weights . 17 

Table  2.5.  Award  Descriptives  by  Accession  Option . 17 

Table  2.6.  Separation  Status  by  Accession  Option . 19 

Table  2.7.  Summary  of  Linkage  Results . 23 

Table  2.8.  Criterion  Variable  Descriptive  Statistics  and  Intercorrelations . 24 

Table  3.1.  Incremental  Validity  Results  of  the  Continuous  Criteria  for  Enlistment  Option 

Sample . 31 

Table  3.2.  Incremental  Validity  Results  of  the  Dichotomous  Criteria  for  Enlistment 

Option  Sample . 32 


x 


CONTENTS  (continued) 


Page 


Table  3.3.  Incremental  Validity  Results  of  the  Continuous  Criteria  for  the  In-Service 

Sample . 35 

Table  3.4.  Incremental  Validity  Results  of  the  Dichotomous  Criteria  for  the  In-Service 

Sample . 36 

Table  3.5.  Incremental  Validity  of  the  OBEF  Predictor  composite  for  In-Service  and 
Enlistment  Option  Incremental  Validity  of  the  OBEF  Predictor  Composites 
for  the  In-Service  and  Enlistment  Option  Samples . 39 

Table  4.1.  Description  of  Longitudinal  Variables . 41 

Table  4.2.  Stage  1  Latent  Growth  Model  Results . 47 

Table  4.3.  Latent  Growth  Modeling  Result  Assessing  Difference  in  OCS  Accession 

Option . 48 

Table  4.4.  Latent  Growth  Results  of  Predictors  and  Outcomes  of  Change . 50 

Table  5.1.  Longitudinal  Milestones  for  the  2008,  2010,  and  2011  OCS  Cohorts . 57 

Table  A.l.  Description  of  Core  Rational  Biodata  Inventory  (RBI)  Component  Scales . A-2 

Table  A.2.  Description  of  Core  Work  Values  Component  Scales . A-2 

Table  D.l.  Full  Information  Maximum  Likelihood  (FIML)  Bivariate  Correlations  by  In- 

Service  and  Enlistment  Options . D-l 


LIST  OF  FIGURES 


Figure  2.1.  Early  Officer  Career  Milestones  for  OCS  2008  Cohort  and  Corresponding 

Data  Collections . 8 

Figure  4. 1 .  Latent  Growth  Model  of  Change  in  Commitment  with  Core  RBI  as  a 

predictor  of  change,  and  Separation  as  an  outcome  of  change . 43 

Figure  4.2.  Latent  Growth  Model  of  Affective  Commitment  for  In-Service  and 

Enlistment  Option  Samples . 45 

Figure  4.3.  Latent  Growth  Model  of  Continuance  Commitment  for  In-Service  and 

Enlistment  Option  Samples . 45 

Figure  4.4.  Latent  Growth  Model  of  Career  Intentions  for  In-Service  and  Enlistment 

Option  Samples . 46 

Figure  4.5.  Latent  Growth  Model  of  Satisfaction  with  Branch  Assignment  for  In-Service 

and  Enlistment  Option  Samples . 46 


xi 


xii 


LONGITUDINAL  VALIDATION  OF  NON-COGNITIVE  OFFICER  SELECTION 
MEASURES  FOR  THE  U.S.  ARMY  OFFICER  CANDIDATE  SCHOOL  (OCS) 


CHAPTER  1:  INTRODUCTION 

Mark  C.  Young 
U.S.  Army  Research  Institute 

The  project  described  in  this  report  is  one  component  of  a  broader,  multi-year  effort  by 
the  U.S.  Anny  Research  Institute  (ARI)  to  identify,  develop,  and  validate  personnel  tests  for  use 
in  officer  selection  and  branch  assignments.  The  primary  goal  of  this  program  is  to  recommend 
and  implement  such  measures  that  will  serve  to  enhance  both  officer  performance  and  career 
continuance  across  commissioning  sources.  The  current  effort  focuses  specifically  on  procedures 
for  selecting  candidates  for  Officer  Candidate  School  (OCS). 

In  this  chapter,  we  provide  general  background  on  the  role  of  OCS  as  a  commissioning 
source  for  Army  officers,  and  describe  ARI’s  initial  effort  to  validate  selection  measures  for 
OCS  candidates.  Next,  we  highlight  the  primary  objectives  of  the  current  effort  and  discuss  our 
research  approach  and  associated  challenges.  Finally,  we  provide  a  brief  overview  of  the 
remaining  chapters. 


Background 

Officer  Candidate  School:  Background  and  Context 

OCS  is  an  important  commissioning  source  for  Army  officers.  OCS  graduates  are 
commissioned  as  second  lieutenants  (paygrade  0-1),  an  entry-level  officer  rank.  Historically,  the 
Anny  has  used  OCS  to  fill  accession  gaps  in  officer  requirements  when  other  commissioning 
sources  were  unable  to  meet  force  structure  needs.  Compared  to  the  Reserve  Officer  Training 
Corps  (ROTC)  and  the  U.S.  Military  Academy  (USMA),  OCS  is  flexible  enough  to  increase  or 
decrease  its  rate  of  commissioning  officers  on  short  notice.  This  is  largely  due  to  the  relatively 
short  duration  of  the  OCS  pre-commissioning  program.  Particularly  in  wartime,  the  Army  must 
make  the  most  of  all  officer  accessioning  sources  to  attract  officer  candidates  with  strong 
propensities  for  retention.  To  the  extent  possible,  it  must  also  offset  any  projected  officer 
shortages. 

When  ARI’s  OCS  selection  research  began  in  2008,  there  had  been  a  large  surge  in  the 
relative  numbers  of  OCS-commissioned  accessions  as  the  size  of  the  Army  was  growing. 
Historically,  these  officers  have  accounted  for  only  about  10%  of  new  officer  accessions. 
However,  their  annual  accession  proportion  grew  to  a  high  of  40%  in  2008,  temporarily  making 
OCS  the  largest  commissioning  source  in  the  Anny1  (Wardynski,  Lyle,  &  Colarusso,  2009). 


1  At  the  time  of  this  writing  in  fiscal  year  2012,  the  Army  was  downsizing,  and  the  proportion  of  OCS  commissions 
had  dropped  to  approximately  20%  of  all  new  officer  accessions.  This  annual  accession  proportion  is  projected  to 
decrease  further  in  fiscal  year  2013. 
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Another  way  in  which  OCS  has  responded  to  Army  mission  needs  was  by  adding  and 
expanding  the  use  of  an  alternative  avenue  for  entry  into  OCS — the  enlistment  option  program. 
Under  this  program,  civilians  with  a  college  degree  can  enter  OCS  after  completing  Basic 
Combat  Training.  The  program  supplements  the  traditional  route  to  OCS  -  the  in-service 
program  -  in  which  Soldiers  (typically  Non-Commissioned  Officers  [NCOs])  can  apply  and  be 
selected  for  OCS  participation.  The  distinction  between  the  two  avenues  for  entry  into  OCS  is 
important,  because  the  officers  commissioned  through  these  programs  tend  to  differ  significantly 
in  terms  of  demographics,  temperament,  and  cognitive  aptitude  (Russell  &  Tremble,  2011).  The 
analyses  conducted  in  2008  also  distinguished  enlistment  option  candidates  with  no  prior 
military  service  from  those  who  had  prior  service  experience.  The  participants  with  prior  military 
service  were  referred  to  as  “Hybrids”  to  reflect  characteristics  shared  by  both  in-service  and 
enlistment  option  samples.2 

With  the  growth  of  OCS  as  a  commissioning  source,  it  was  especially  important  to 
examine  the  OCS  selection  process,  and  to  determine  what  types  of  attributes  could  be  used  to 
select  those  individuals  who  would  most  likely  develop  into  highly  qualified  officers  motivated 
to  pursue  an  Army  career.  Currently,  with  the  exception  of  the  Anned  Services  Vocational 
Aptitude  Battery  (ASVAB)  General  Technical  (GT)3  score  and  a  passing  score  on  the  Army 
Physical  Fitness  Test  (APFT),  the  OCS  selection  process  has  relied  largely  on  qualitative  and 
descriptive  materials  such  as  interviews,  documentation  of  educational  attainments, 
recommendations  from  superiors,  and  essays  asking  individuals  why  they  want  to  become  Anny 
officers.  The  need  for  larger  numbers  of  officers  to  be  commissioned  through  OCS  prompted 
ARI  to  review  the  OCS  selection  process  and  to  consider  how  selection  measures  and 
implementation  procedures  could  be  optimized.  This  was  done  in  support  of  meeting  the  Army’s 
future  OCS  accessioning  requirements  for  high-perfonning  officers. 

Preliminary  OCS  Research  Effort 

The  first  phase  of  the  OCS  research  (Russell  &  Tremble,  2011)  served  as  the  foundation 
for  the  effort  described  in  this  report.  This  phase  began  in  2008  when  ARI  administered  an 
extensive  battery  of  personnel  selection  tests  to  over  1,300  entering  OCS  candidates.  The  test 
battery,  the  Officer  Background  and  Experiences  Fonn  (OBEF),  included  measures  of 
temperament,  affectivity,  values,  and  leadership.  One  of  these  measures  was  a  version  of  the 
Rational  Biodata  Inventory  (RBI).  The  RBI  is  a  non-cognitive  measure  of  personality  and 
motivational  attributes  that  has  shown  promise  for  enhancing  selection  of  cadets  in  ROTC 
(Putka,  Wasko,  Tremble,  Kilcullen,  &  Shaw,  2009;  Wasko,  Putka,  Legree,  &  Kilcullen,  2011) 
and  enlisted  Soldiers  (Knapp  &  Heffner,  2009;  Knapp,  Owens,  &  Allen,  2011).  Instruments 
measuring  work  values  and  judgment  in  leadership  situations  were  also  administered  as  part  of 
the  OBEF.  We  used  a  longitudinal  validation  research  design  to  track  these  tested  candidates 
through  the  end  of  their  12-week  OCS  training  course.  In  addition,  approximately  50%  of  these 


2  Officers  with  enlisted  service  from  another  branch  of  the  Armed  Forces  (e.g.,  Navy,  Air  Force)  were  also  included 
in  the  Flybrid  group. 

3  In  2008,  researchers  were  unable  to  obtain  GT  scores  for  the  full  sample  of  respondents  (Russell  &  Tremble, 
2011).  For  this  reason,  the  Armed  Forces  Qualification  Test  (AFQT) — a  composite  of  four  individual  ASVAB 
tests —  was  used  as  a  proxy  for  GT  in  2008  and  in  the  present  analysis. 
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individuals  were  tracked  further  into  their  Basic  Officer  Leadership  Course  III  (BOLC  III,  now 
referred  to  as  BOLC  B),  where  newly  commissioned  officers  complete  their  branch-specific 
training.  Data  collected  both  at  the  end  of  OCS  and  during  BOLC  III/B  (approximately  6  months 
following  OCS  graduation)  were  used  to  obtain  early  indicators  of  perfonnance  and  continuance 
(e.g.,  OCS  training  perfonnance  scores,  self-reported  commitment,  and  career  continuance 
intentions)  for  the  longitudinal  validation. 

The  preliminary  findings  from  this  first  phase  of  the  OCS  research  showed  that  a  subset 
of  the  OBEF  measures  was  predictive  of  practical  and  conceptually  important  outcomes  such  as 
commitment  to  the  Anny,  career  continuance  intentions,  and  OCS  perfonnance  scores  (Russell 
&  Tremble,  2011;  Young,  Ardison,  &  Babin,  2011).  Importantly,  these  tests  demonstrated 
incremental  validity  beyond  that  of  the  Armed  Forces  Qualification  Test  (AFQT)  (a  composite 
derived  from  the  ASVAB),  and  the  findings  were  similar  for  the  enlistment  option,  in-service, 
and  hybrid  subsamples.  Similar  validity  findings  were  obtained  from  two  independent  samples  of 
OCS  candidates  tested  in  2009  and  2011  and  subsequently  tracked  to  obtain  initial  criterion  data 
(Allen,  Bynum,  Erk,  Babin,  &  Young,  2012;  Russell,  Allen,  &  Babin,  2011). 

OCS  Longitudinal  Validation  Research 

The  effort  described  here  was  supported  by  the  Officer  Candidate  School  (OCS;  Fort 
Benning,  Georgia)  and  the  Office  of  the  Anny  G-l .  The  work  was  conducted  under  contract  with 
the  Human  Resources  Research  Organization  (HumRRO).  ARI  worked  in  close  collaboration 
with  HumRRO  throughout  this  project. 

When  the  project  began  in  early  2011,  the  junior  officers  in  the  2008  OCS  cohort  were 
approaching  the  end  of  their  3-year  Active  Duty  Service  Obligation  (ADSO).  This  was  a  critical 
period  as  officers  were  deciding  whether  or  not  to  continue  serving  in  the  Regular  Anny.  At  that 
time,  criterion  information  on  these  individuals  had  not  been  collected  since  the  BOLC  III/B  data 
collection  described  above.  The  maturing  of  this  longitudinal  officer  sample  offered  an  important 
opportunity  to  capture  longer-term  career  outcomes  (i.e.,  performance  and  continuance)  for  use 
in  validating  the  OBEF  selection  battery  administered  in  2008.  The  goal  was  to  advance  beyond 
our  current  and  limited  knowledge  of  the  OBEF’s  validity,  by  validating  the  battery  against  an 
expanded  set  of  longer-term  criteria. 

Objectives 

The  specific  objectives  of  this  effort  were  to  (a)  collect  longer-term 4  criterion  measures 
(e.g.,  in-unit  perfonnance,  service  continuance)  for  the  members  of  our  2008  OCS  cohort,  (b) 
validate  the  2008  test  battery  (the  OBEF)  against  these  longer-tenn  outcomes,  and  (c)  validate 
the  OCS  composites  developed  in  the  first  phase  against  the  same  long-term  outcomes. 
Additionally,  the  data  collections  described  above  offer  a  unique  opportunity  to  examine  changes 
in  officer  attitudes  (e.g.,  commitment  to  the  Army)  over  time.  Thus,  a  fourth  objective  was  to  (d) 


4  We  use  the  term  “longer-term”  in  referencing  career  outcomes  that  go  beyond  the  point  of  OCS  training  but  are 
still  relatively  early  in  the  overall  officer  career  life -cycle. 
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model  changes  in  officer  attitudes  during  their  initial  ADSO,  and  the  antecedents  and 
consequences  of  those  changes. 

These  objectives  were  driven  in  large  part  by  a  need  to  transition  our  experimental  OCS 
selection  tests  to  operational  use  in  an  Initial  Operational  Test  and  Evaluation  (IOT&E)  within 
the  next  few  years.  The  current  research  was  designed  to  help  inform  decisions  regarding  the 
choice  of  the  operational  test  battery  and  the  Army’s  officer  candidate  selection  policies. 

Challenges 

In  accomplishing  these  objectives,  there  was  an  urgency  to  obtain  empirically-supported 
evidence  to  inform  our  short-term  recommendations  for  enhancing  officer  selection  measures 
and  related  policies.  This  sense  of  urgency  was  driven  by  previous  research  suggesting  that  the 
potential  benefits  of  refining  the  Anny’s  selection  procedures  were  considerable  (e.g.,  Russell  & 
Tremble,  2011),  and  would  last  for  many  years  beyond  the  point  of  the  selection  process. 
However,  the  junior  officers  in  our  2008  OCS  cohort  had  only  been  commissioned  for 
approximately  3  years;  therefore  no  opportunity  existed  to  validate  our  new  selection  tests 
against  “long-term”  performance  and  continuance  outcomes  in  this  sample.  Ideally,  such  long¬ 
term  outcomes  (e.g.,  performance  as  a  mid-career  officer,  career  continuance  behavior  observed 
10  years  beyond  commissioning)  would  be  used  for  the  validation.  This  research  constraint  has 
been  a  recurring  challenge  throughout  the  OCS  research  program.  For  this  reason,  “interim” 
criterion  measures  (e.g.,  training  performance,  self-reported  career  continuance  intentions)  have 
been  obtained  whenever  possible.  These  interim  criterion  measures  help  achieve  key  research 
objectives  while  longer-tenn  outcome  data  (e.g.,  actual  continuance  well  beyond  the  initial 
ADSO)  becomes  available  for  the  maturing  OCS  research  samples. 

Another  challenge  for  this  effort  has  been  the  need  to  obtain  criterion  data  on  the  2008 
OCS  cohort  as  they  have  become  increasingly  dispersed  across  locations.  During  OCS  training, 
all  cohort  members  were  at  the  same  location,  making  it  relatively  easy  and  cost  effective  to 
collect  both  self-report  and  administrative  (e.g.,  training-related)  records.  However,  once  they 
completed  their  initial  training  (through  BOLC  III/B)  and  reached  their  first  unit  of  assignment, 
they  were  spread  out  across  the  globe.  This  made  the  data  collections  even  more  challenging, 
especially  given  the  wartime  environment  (with  frequent  deployments)  and  organizational 
constraints  regarding  access  to  Anny  personnel  for  research  support. 

Developing  criterion  measures — along  with  the  methods  of  capturing  them  for  the  target 
sample — has  been  a  major  focus  of  the  current  effort.  Throughout  the  project,  the  most  feasible 
and  relevant  outcome  information  given  the  sample,  logistical,  cost,  and  organizational 
constraints  was  obtained. 

Organization  of  Report 

Chapter  2  (Research  Methodology)  presents  the  general  research  approach  and  provides  a 
brief  description  of  the  Officer  Background  and  Experiences  Form  (OBEF)  selection  battery 
administered  to  the  OCS  candidates  in  2008.  After  describing  the  research  sample,  the  criterion 
measures  (both  self-report  and  administrative)  are  discussed  in  detail. 
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Chapter  3  (Prediction  of  Officer  Perfonnance  and  Continuance  with  the  OBEF)  begins  by 
summarizing  the  original  validity  findings  from  the  first  phase  of  this  research  (Russell  & 
Tremble,  2011).  Next,  the  validation  approach  for  the  new  analyses  is  described,  followed  by  a 
presentation  of  the  incremental  validity  results  for  both  in-service  and  enlistment  option  program 
participants.  These  results  show  the  relative  predictive  validity  of  the  various  tests  within  the 
selection  battery  across  criterion  measures  for  the  two  subgroups. 

Chapter  4  (Longitudinal  Examination  of  Officer  Attitudes  over  Time)  reports  findings 
from  a  longitudinal  analysis  of  self-reported  commitment  and  career  continuance  intentions  for 
in-service  and  enlistment  option  candidates.  The  findings  are  based  on  a  Latent  Growth  Curve 
Modeling  (LGM)  procedure  with  repeated  measures  collected  over  a  3 -year  period.  The 
relationships  between  attitude  change  and  subsequent  officer  career  outcomes  (e.g.,  actual 
separation)  also  are  examined. 

Chapter  5  (Conclusions  and  Next  Steps)  summarizes  the  key  findings  and  discusses  their 
practical  implications  for  moving  ARI’s  experimental  OCS  selection  tests  toward 
implementation.  This  chapter  provides  specific  recommendations  regarding  what  predictor 
measures  should  be  in  the  final  operational  selection  battery  and  what  testing  policies  and 
procedures  would  help  to  optimize  its  utility.  Finally,  the  limitations  of  the  current  effort  are 
noted,  along  with  recommendations  regarding  next  steps  for  addressing  these  issues. 
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CHAPTER  2:  RESEARCH  METHODOLOGY 


Matthew  T.  Allen  &  Joy  T.  Oliver 
Human  Resources  Research  Organization 

Mark  C.  Young 
U.S.  Army  Research  Institute 

Teresa  L.  Russell  &  Bethany  H.  Bynum 
Human  Resources  Research  Organization 

Sharon  Meyers 
U.S.  Army  Research  Institute 

The  purpose  of  this  chapter  is  to  describe  the  research  methodology,  with  particular 
emphasis  on  the  procedures  and  measures  administered  to  the  2008  Officer  Candidate  School 
(OCS)  sample  approximately  three  years  after  completing  OCS.  We  begin  with  an  overview  of 
the  research  design,  including  a  description  of  the  career  progression  of  the  OCS-commissioned 
officers  in  our  sample.  This  is  followed  by  a  brief  description  of  the  predictor  measures 
administered  in  2008.  Next,  we  describe  the  development  of  the  “In-Unit”  survey  administered 
to  the  2008  OCS  cohort.  We  then  provide  a  more  detailed  discussion  of  the  data  obtained  on  the 
research  participants,  which  includes  both  administrative  personnel  records  and  self-report 
measures  collected  by  the  research  team.  We  conclude  the  chapter  with  a  description  of  the 
criterion  measures. 


Research  Overview 

We  collected  non-cognitive  predictor  data  from  over  1,300  officer  candidates  across  10 
consecutive  OCS  classes  in  2008.  OCS  is  a  12-week  course  covering  physical  fitness,  leadership, 
and  academic  content.  After  successful  completion  of  OCS,  graduates  are  commissioned  as 
Second  Lieutenants  and  attend  the  Basic  Officer  Leadership  Course  (BOLC).  OCS  graduates 
from  the  earlier  classes  attended  BOLC  II,  a  7-week  course  covering  general  officer  skills  such 
as  leadership,  and  BOLC  III,  a  branch-specific  course  that  varied  in  length.  Some  of  the  later 
graduates  in  the  cohort  attended  BOLC  B,  a  merged  form  of  BOLC  II  and  BOLC  III,  after 
BOLC  II  was  discontinued  in  December  2009  (Cavallaro,  2009). 

For  most  officers,  promotion  to  first  lieutenant  (1LT)  occurs  when  the  officer  has  18 
months  of  time  in  service  (from  the  date  of  acceptance  of  the  commission).  Promotion  to  1LT  is 
automatic,  except  in  rare  instances  when  an  officer  is  under  investigation  or  pending  dismissal. 
For  the  2008  OCS  sample,  this  promotion  would  have  occurred  in  early  to  mid-2010.  OCS 
graduates  incur  a  three-year  Active  Duty  Service  Obligation  (ADSO),  after  which  they  can  join 
the  Individual  Ready  Reserve  (IRR),  an  Anny  National  Guard  (ARNG)  unit,  the  Anny  Reserve 
(USAR),  or  choose  to  remain  on  active  duty.  For  the  2008  sample,  we  estimated  the  completion 
of  this  obligation  would  occur  between  the  summer  of  201 1  and  early  2012. 
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Beyond  their  3 -year  ADSO,  for  officers  that  remain  on  active  duty,  promotion  to  captain 
typically  occurs  at  about  4  years  time  in  service.  Promotion  to  captain  is  detennined  by  the 
Department  of  the  Anny,  but  the  selection  rate  is  exceptionally  high.  The  next  formal  step  in  an 
officer’s  education  is  the  Captain’s  Career  Course  (CCC)  which  is  open  to  senior  first  lieutenants 
and  Army  captains  with  4-7  years  of  military  experience.  Like  BOLC  III,  CCC  is  branch-specific 
and  varies  in  length  and  substance.  Upon  graduation,  officers  transition  to  their  next  duty 
assignment  as  a  company  commander,  battalion  staff  officer,  brigade  staff  officer,  or  a  similar 
position. 

ARI’s  program  of  OCS  research  is  designed  to  capture  information  regarding  officer 
attitudes,  retention,  and  performance  at  key  points  throughout  their  Army  career.  The 
correspondence  between  these  key  points  and  the  data  collections  are  illustrated  in  Figure  2.1. 
Specifically,  the  following  infonnation  was  collected  from  the  2008  sample  at  the  various  points 
in  time: 

1 .  Time  1:  Beginning  of  OCS  -  Within  their  first  week  of  OCS,  participants  from  10 
consecutive  classes  completed  the  Officer  Background  and  Experiences  Form 
(OBEF),  a  battery  of  non-cognitive  measures. 

2.  Time  2:  End  of  OCS  -  About  10  weeks  after  the  beginning  of  OCS  data  collection 
(near  the  end  of  OCS),  participants  completed  a  short  survey  regarding  their  attitudes 
towards  the  Anny  and  continuance  intentions.  Perfonnance  information  (i.e.,  Order  of 
Merit  List  [OML]  scores;  Russell  &  Tremble,  2011)  was  collected  on  these 
participants  from  OCS  administrative  records. 

3.  Time  3:  BOLC  III/B  -  Approximately  six  months  after  completing  OCS  (near  the 
anticipated  end  of  BOLC  III/B),  participants  completed  a  different  (though  similar  to 
Time  2)  survey  of  their  attitudes  and  continuance  intentions.  Collecting  information  at 
this  point  was  critical  because  it  was  the  first  time  the  participants  received  training  in 
their  branch-specific  duties.  Infonnation  regarding  whether  they  recycled  during 
BOLC  II  or  BOLC  III  (or  BOLC  B)  was  collected  from  administrative  records. 

4.  Time  4:  In-Unit  -  Near  the  completion  of  their  ADSO,  participants  were  asked  to 
complete  a  final  measure  of  their  attitudes  and  continuance  intentions.  This  survey 
had  overlapping  content  with  the  Time  2  and  Time  3  surveys,  but  also  asked 
participants  to  self-report  indices  of  perfonnance  in  their  role  as  U.S.  Anny  officers. 

The  purpose  of  this  research  was  to  validate  the  OBEF  against  longer-term  criteria. 
Accordingly,  only  the  OBEF  predictor  measures  (from  Beginning  of  OCS,  Time  1),  the  In-Unit 
criterion  instrument  (from  Time  4),  and  the  perfonnance  and  retention  criterion  data  from 
administrative  records  are  described  in  any  detail  in  this  report.  Readers  interested  in  more 
details  for  the  End  of  OCS  (Time  2)  data  collections  should  consult  Babin,  DiFazio,  Allen,  and 
Hooper  (2011)  for  procedural  information  and  Allen,  Campbell,  and  Kilcullen  (2011)  regarding 
the  End  of  OCS  survey.  Readers  interested  in  the  BOLC  III/B  survey  should  contact  ARI. 
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Data  Collections  Early  Career  Milestones 


Officer  Background  and  Experiences  Form  (OBEF) 


The  OBEF  is  a  battery  of  instruments  designed  to  assess  individuals’  temperament, 
affectivity,  values,  and  judgment.  One  of  four  versions  of  the  OBEF  was  administered  across  10 
consecutive  classes  to  the  2008  OCS  cohort.  Each  of  these  versions  contained  different 
combinations  of  unique  and  overlapping  content5  (see  Russell  &  Tremble,  2011  for  more 
details).  However,  a  “core”  set  of  instruments  and  scales  was  (a)  common  across  multiple  OBEF 
versions  and  (b)  exhibited  the  most  potential  for  selecting  candidates  into  OCS.  The  core  set  of 
instruments  and  scales  can  be  summarized  in  the  following  three  groups: 

1 .  Core  Rational  Biodata  Inventory  (RBI)  -  The  RBI  is  a  theoretically-keyed  biodata 
instrument  containing  questions  about  previous  experiences,  thoughts,  opinions,  and 
actions  (Kilcullen,  Tremble,  &  Robbins,  2011).  Each  item  contains  a  statement  (e.g., 
“How  often  have  you  done  X  in  the  past”)  and  five  response  options  (e.g.,  “Very  Often” 
to  “Never”).  The  RBI  is  scored  by  computing  an  arithmetic  average  of  a  scale’s 
component  items.  The  core  scales  measure  nine  dimensions  of  individual  temperament: 
Peer  Leadership,  Achievement,  Fitness  Motivation,  Self-Efficacy,  Stress  Tolerance, 
Hostility  to  Authority,  Army  Affective  Commitment,  Continuance  Commitment,  and 
Traditional  Values.  The  coefficient  alphas  for  the  component  scales  were  generally 
acceptable  for  biodata  instruments,  ranging  from  .61  to  .78,  though  the  alpha  for  the 
Hostility  to  Authority  scale  was  .58.  Despite  its  lower  reliability,  we  included  this  scale 
in  subsequent  analysis  due  to  ambiguities  in  the  interpretation  of  internal  consistency 
estimates  for  scales  based  on  biodata  items  (Bollen  &  Lennox,  1991). 

2.  Core  Work  Values  -  The  Work  Values  instrument  asks  respondents  to  rate  how 
important  various  aspects  of  a  job  (e.g.,  “Having  a  flexible  work  schedule”)  are  to  them 
personally  on  a  1  (“Not  at  all  important”)  to  5  (“Extremely  important”)  scale  (Robbins, 
Allen,  &  Putka,  2011).  The  Work  Values  scales  are  scored  by  computing  an  arithmetic 
average  of  a  scale’s  component  items.  The  Core  Work  Values  scales  assess  the 
importance  of  seven  core  work  preferences  to  each  individual:  Rugged  Leadership, 
Teamwork,  Flexibility/Choice,  Structure/Recognition,  Altruism/Benevolence,  Skill 
Development,  and  Job  Security.  The  coefficient  alphas  for  these  scales  ranged  from  .70 
to  .90. 

3.  Situational  Judgment  Test  (SJT)  -  The  SJT  was  developed  using  items  from  previous 
tests  to  assess  an  individual’s  judgment  in  leadership  situations.  Respondents  are  asked 
to  read  a  brief  description  of  a  situation  typically  encountered  by  Army  officers,  then  rate 
the  effectiveness  of  four  courses  of  action  on  a  1  (“Ineffective”)  to  7  (“Very  effective”) 
scale.  Though  multiple  scoring  algorithms  were  tested  in  the  original  OCS  research 
(Pearlman,  Allen,  Putka,  Hooper,  &  Waters,  2011),  the  scores  derived  from  the 
“Standardized  Subject  Matter  Expert  (SME)-Keyed”  method  was  used  in  the  present 
research.  This  method  involves  using  SMEs  to  detennine  the  appropriate  “effectiveness” 
of  each  action  and  the  subsequent  application  of  profde  scoring  (McDaniel,  Psotka,  & 

5  Content  included  in  most  versions  of  the  OBEF  was  considered  “core,”  while  the  remaining  content  was 
considered  “experimental.”  Due  to  sample  size  issues,  and  for  the  sake  of  parsimony  in  analyzing  the  data  and 
writing  the  report,  subsequent  analyses  only  involve  the  “core”  scales. 
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Legree,  2009).  While  the  coefficient  alpha  for  the  SJT  using  this  scoring  method  was 

low  (a  =  .54),  it  resulted  in  lower  subgroup  differences  and  higher  validity  coefficients 

than  other  scoring  approaches  in  the  2011  analysis. 

Descriptions  of  the  component  scales  for  the  RBI  and  Work  Values  instruments  can  be 
found  in  Appendix  A.  Scales  from  the  RBI  and  Work  Values  instruments  were  chosen  to  form 
two  composites  (one  for  in-service  and  one  for  enlistment  option  OCS  applicants)  designed  to 
predict  officer  perfonnance  (in  OCS)  and  retention. 

In-Unit  (Time  4)  Survey  Development 

Collecting  meaningful  perfonnance-related  information  on  the  2008  OCS  cohort  became 
much  more  difficult  as  they  progressed  beyond  the  point  of  their  initial  officer  training.  While 
attending  OCS,  these  cohort  members  were  all  at  the  same  location  and  training  perfonnance 
infonnation  was  readily  accessible.  However,  after  completing  branch  training  (BOLC  III/B)  and 
reaching  their  initial  unit  of  assignment,  the  officers  were  spread  out  to  locations  across  the 
globe.  This  dispersion  made  it  infeasible  to  collect  outcome  measures  through  “on  site”  data 
collections — for  both  cost  and  logistical  reasons.  In  addition,  though  some  useful  infonnation 
can  be  captured  from  automated  personnel  records  (see  later  in  this  chapter),  the  amount  of 
perfonnance-related  information  covering  this  point  in  an  officer’s  career  is  generally  limited 
because  such  records  are  intended  for  administrative  purposes  and  not  for  personnel  research. 
Given  these  constraints  to  collecting  in-unit  criterion  data,  it  was  concluded  that  a  web-based 
self-report  measure  would  provide  the  most  feasible  option  for  capturing  most  of  the  officer 
outcome  infonnation  needed  for  this  research. 

The  In-Unit  Survey  was  designed  to  assess  officers’  attitudes,  retention  intentions,  and 
perfonnance  both  in  training  and  in  unit.  Demographic  and  background  infonnation  thought  to 
be  predictive  of  officer  retention  was  also  collected.  The  In-Unit  Survey  was  created  using  items 
derived  from  previous  ARI  research  efforts,  such  as  the  Career  History  Survey  from  the  Army 
Officer  Predictor  and  Criterion  project  (Paullin  et  ah,  2012),  the  Survey  on  Officer  Careers 
(SOC;  e.g.,  Jones,  1999),  and  other  OCS  research  (e.g.,  Russell  et  al.,  2011).  New  items  tailored 
specifically  for  our  sample  were  also  created  by  addressing  career  continuance  intentions  and 
behaviors  that  are  especially  relevant  to  officers  reaching  the  end  of  their  ADSOs.  The  survey 
was  divided  into  two  major  sections  with  respective  subsections: 

1 .  Background 

A.  Overview 

B.  Civilian  and  Military  Education/Training 

C.  Branch  and  Assignments 

D.  Deployment  History 

2.  Perfonnance 

A.  In-Unit  Performance 

B.  Career  Intent  and  Attitudes 

Background  Section.  The  initial  portion  of  the  Background  section  was  comprised  of  4 
overview  questions  used  for  identification  purposes:  (Ql)  Army  component,  (Q2)  current  pay 
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grade,  (Q3)  marital  status,  and  (Q4)  number  of  dependent  children.  Questions  1  and  2  were 
adapted  from  the  Career  History  Survey,  while  Questions  3  and  4  were  adapted  from  ARI’s 
SOC,  a  longitudinal  survey  of  the  population  of  Army  officers  that  typically  occurs  every  two 
years.  In  addition  to  collecting  this  infonnation  for  identification  purposes,  these  data  were 
collected  because  SOC  research  has  indicated  that  these  demographic  variables  (e.g.,  marital 
status  and  number  of  dependent  children)  are  related  to  morale  and  satisfaction  with  the  Army  as 
a  career  (HumRRO  &  Jones,  1999). 

The  “Civilian  and  Military  Education/Training”  comprised  the  second  part  of  the 
Background  section.  Questions  were  derived  from  a  combination  of  SOC  items  (e.g.,  GPA  and 
level  of  educational  attainment),  the  Career  History  Survey  (i.e.,  undergraduate  GPA;  Paullin  et 
ah,  2012),  the  Educational  Attainment  Survey  administered  by  the  Census  Bureau  (e.g., 
undergraduate  college  major),  and  a  BOLC  III/B  survey  used  in  other  ARI  research6  (i.e., 
questions  related  to  recycling  in  OCS  and/or  BOLC  II/III/B  and  the  respective  reasons; 
performance  in  BOLC  II/III/B). 

The  “Branch  and  Assignments”  subsection  was  comprised  of  questions  related  to  the  branch 
to  which  officers  were  commissioned,  when  the  branch  was  assigned  and  how  the  branch  was 
detennined,  as  well  as  questions  regarding  whether  the  commissioning  branch  was  one  of  the 
officer’s  top  three  choices.  These  questions  were  adapted  primarily  from  a  BOLC  III/B  survey  used 
in  other  ARI  research. 

Finally,  the  Background  section  concluded  with  questions  related  to  the  2008  OCS  cohort’s 
assigmnents.  Specific  positions  are  theorized  to  relate  to  tenure  and  job  perfonnance  criteria  (e.g., 
promotion  rate);  however,  there  is  currently  little  empirical  evidence  for  these  relationships.  The 
questions  comprising  the  “Deployment  History”  section  were  drawn  from  the  Career  History  Survey, 
and  were  administered  because  of  their  relationship  with  tenure  and  retention.  An  example  question 
is  “Have  you  been  deployed  since  becoming  a  commissioned  officer?” 

Performance  Section.  The  “In-Unit  Performance”  section  was  comprised  of  both 
verifiable  and  subjective  questions  regarding  officers’  performance  since  receiving  their 
commission.  Verifiable  perfonnance  was  determined  by  asking  officers  about  (a)  awards  and 
decorations,  (b)  disciplinary  actions,  and  (c)  ratings  on  their  most  recent  Officer  Evaluation 
Record  (OER).  These  items  were  adapted  from  the  BOLC  III/B  (Time  3)  survey  and  Career 
History  Survey.  In  addition  to  their  intended  use  as  test  validation  criteria,  these  perfonnance 
questions  are  of  additional  interest  because  of  their  potential  for  predicting  long-tenn 
perfonnance  and  continuation  of  service  beyond  the  ADSO.  The  verifiable  perfonnance 
questions  were  followed  by  more  specific  questions  asking  officers  to  provide  their  subjective 
evaluations  of  their  own  perfonnance  in  comparison  to  those  of  their  peers  in  the  following 
areas: 


•  Physical  Fitness,  Health,  and  Endurance  -  Effectively  engages  in  the  Anny  physical 
readiness  training  program  both  as  a  leader  and  a  participant.  Meets  height  and  weight 


6  Note  that  this  is  not  the  same  survey  as  the  BOLC  III/B  (Time  3)  survey  analyzed  in  the  present  report. 
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standards  and  passes  the  Anny  Physical  Fitness  Test.  Successfully  completes  all  physical 
and  endurance  requirements  of  platoon  commander  or  equivalent  position. 

•  Anny  Values,  Warrior  Ethos,  and  Ethics  -  Embodies,  lives,  and  defends  Army  Values 
and  Warrior  Ethos.  Adheres  to  Army  officer  standards  of  conduct  and  holds  to  a  highly 
ethical  code  of  behavior. 

•  Technical  Competence  -  Effectively  demonstrates  technical  skills  for  performing  branch 
integration  of  requirements  as  a  member  of  the  combined  arms  team. 

•  Tactical  Competence  -  Effectively  employs  Warrior  Tasks  and  Battle  Drills,  branch 
technical  skills,  and  makes  necessary  contributions  to  solving  tactical  problems. 

•  Leadership  -  Effectively  performs  as  an  Army  leader  in  all  positions  and  assignments. 
Applies  Troop  Leading  Procedures  (TLP)  in  execution  of  leadership  roles.  Supports 
subordinate,  peer,  and  superior  leaders. 

•  Officership  -  Inculcates  the  characteristics  of  being  an  officer  to  include  customs, 
traditions,  and  heritage.  Develops  knowledge  and  awareness  of  the  Army  as  an  institution 
to  include  military  operation  theory  and  management,  personnel  and  career  management, 
military  justice  and  discipline. 

Participants  were  asked  to  rate  their  performance  in  comparison  to  their  peers  on  a  1-5  scale, 
where: 

1  =  Well  Below  Average:  Bottom  20%  of  Peer  Group 

2  =  Next  20%  of  Peer  Group 

3  =  Average:  Middle  20%  of  Peer  Group 

4  =  Next  20%  of  Peer  Group 

5  =  Well  Above  Average:  Top  20%  of  Peer  Group 

The  scale  also  included  a  “Not  Applicable/Cannot  Rate”  option. 

Finally,  the  “Career  Intent  and  Attitudes”  items  were  originally  developed  for  the  End  of 
OCS  survey.  These  items  measured  morale  (e.g.,  “How  would  you  rate  your  current  level  of 
morale?”),  commitment  to  the  Anny  (e.g.,  “I  feel  ‘emotionally  attached’  to  the  Army.”),  breach 
of  psychological  contract  (e.g.,  “In  general,  the  Army  has  kept  its  promises  to  me.”),  highest  rank 
estimation  (e.g.,  “What  is  the  highest  rank  you  think  you  will  achieve  in  your  Army  career?”), 
and  general  career  intentions  (e.g.,  “I  plan  to  stay  in  the  Army  until  retirement.”).  Many  of  these 
items  have  been  administered  in  other  phases  of  this  research  effort  (Beginning  of  OCS,  End  of 
OCS,  BOLC  III/B),  and  thus  are  the  subject  of  the  longitudinal  analyses  described  in  Chapter  4. 
Finally,  the  last  two  items  asked  whether  the  officer  requested  a  release  from  active  duty  around 
the  time  of  their  ADSO  completion,  and  if  the  officer  did  plan  to  separate,  to  indicate  why  (e.g., 
“Offered  a  good  civilian  job”).  The  full  In-Unit  Survey  can  be  found  in  Appendix  B. 
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Data  Collection  Procedures 


The  results  presented  in  this  report  are  based  on  (a)  data  collected  from  the  Time  4,  In- 
Unit  Survey,  which  was  administered  online  to  participants  from  the  2008  OCS  cohort,  and  (b) 
administrative  performance  and  retention  data  obtained  from  Army  records.  Efforts  to  capture 
relevant  information  from  other  sources,  such  as  the  participants’  supervisors,  are  ongoing. 

In-Unit  Survey  Administration 

The  In-Unit  Survey  was  sent  to  all  2008  OCS  research  participants  who,  according  to 
administrative  records,  were  still  in  the  Anny,  regardless  of  component  (i.e.,  Regular  Army, 
USAR,  ARNG).  The  solicitation  email  was  sent  to  all  participants  at  once,  regardless  of  when 
they  completed  the  OCS  (Time  1)  data  collection  in  2008.  A  number  of  steps  were  taken  to 
increase  the  response  rate  for  this  data  collection.  First,  a  letter  from  the  OCS  Commandant  at  the 
time  of  the  research  was  included  as  part  of  the  solicitation,  stressing  the  importance  of  the 
research  and  encouraging  the  participants  to  complete  the  instrument.  Second,  the  survey  was 
left  open  for  over  two  months  (July  18,  201 1  to  September  23,  2011),  with  multiple  reminders 
sent  to  non-respondents  over  the  course  of  those  two  months.  This  gave  potential  participants 
multiple  opportunities  to  complete  the  survey. 

Upon  logging  on  to  the  survey,  participants  were  asked  to  read  a  privacy  act  statement 
and  complete  an  informed  consent  form.  If  the  individual  agreed  to  participate,  he  or  she  was 
taken  to  the  actual  survey.  Overall,  543  individuals  responded  to  the  survey,  or  40.4%  of  the 
original  sample.  The  demographics  of  this  sample  are  reported  in  Table  2.1.  Approximately  45% 
of  the  sample  came  to  OCS  through  the  in-service  option,  while  33%  came  through  the 
enlistment  option.  We  classified  the  rest  as  hybrids.  The  sample  was  mostly  male,  non-Hispanic, 
and  either  White  or  African  American.  However,  a  greater  proportion  of  African  Americans 
were  found  in  the  in-service  sample  than  in  the  enlistment  option  sample. 

For  comparison  purposes,  the  demographic  characteristics  for  the  full  2008  sample  are 
presented  in  Table  2.2.  Overall,  the  proportions  of  demographics  for  the  In-Unit  Survey 
respondents  are  similar  to  the  demographic  proportions  for  the  full  sample,  with  some  variation 
within  the  specific  subsamples.  For  example,  a  higher  proportion  of  White  respondents 
participated  in  the  In-Unit  enlistment  option  sample  than  in  the  full  enlistment  option  sample. 
Additionally,  the  Anned  Forces  Qualification  Test  (AFQT)  scores  are  slightly  higher  in  the  In- 
Unit  sample  than  the  full  sample.  As  with  the  full  sample,  AFQT  scores  in  the  In-Unit  sample, 
and  the  enlistment  option  sample  in  particular,  were  highly  range  restricted  (i.e.,  smaller  standard 
deviations)  compared  to  population  estimates  (Russell  &  Tremble,  2011). 
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Table  2.1.  Officer  Candidate  School  (OCS)  Sample  Demographics  for  In-Unit  Survey 
Respondents 


In-Service 
(n  =  243) 

Enlistment 
Option 
(n  =  181) 

Hybrid 
(77  =  103) 

Total 
(N=  543) 

n 

% 

n 

% 

n 

% 

n 

% 

Gender 

Male 

188 

77.4 

143 

79.0 

88 

85.4 

424 

78.1 

Female 

47 

19.3 

29 

16.0 

10 

9.7 

86 

15.8 

Missing  values 

8 

3.3 

9 

5.0 

5 

4.9 

33 

6.1 

Ethnicity 

Hispanic 

52 

21.4 

17 

9.4 

19 

18.4 

89 

16.4 

Non-Hispanic 

181 

74.5 

153 

84.5 

80 

77.7 

418 

77.0 

Missing  values 

10 

4.1 

11 

6.1 

4 

3.9 

36 

6.6 

Race 

White 

127 

52.3 

151 

83.4 

79 

76.7 

359 

66.1 

American  Indian/Alaskan 

2 

.8 

0 

0 

1 

1.0 

3 

0.6 

Asian 

4 

1.6 

9 

5.0 

4 

3.9 

17 

3.1 

Black  or  African  American 

79 

32.5 

8 

4.4 

12 

11.7 

101 

18.6 

Pacific  Islander 

0 

0 

2 

1.1 

1 

1.0 

3 

0.6 

Multiple 

8 

3.3 

6 

3.3 

2 

1.9 

16 

2.9 

Missing  values 

23 

9.5 

5 

2.8 

4 

3.9 

44 

8.1 

Army  Component 

Regular  Army 

234 

96.3 

167 

92.3 

85 

82.5 

500 

92.1 

Army  Reserve 

1 

0.4 

10 

5.5 

10 

9.7 

22 

4.1 

Army  National  Guard 

5 

2.1 

2 

1.1 

5 

4.9 

12 

2.2 

Missing  Values 

3 

1.2 

2 

1.1 

3 

2.9 

9 

1.7 

Pay  Grade 

01 

1 

0.4 

2 

1.1 

2 

1.9 

5 

0.9 

02 

239 

98.4 

175 

96.7 

94 

91.3 

523 

96.2 

03 

1 

0.4 

0 

0.0 

0 

0.0 

1 

0.2 

Missing  Values 

2 

0.8 

4 

2.2 

7 

6.8 

14 

2.6 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

AFQT 

70.86 

19.57 

90.11 

8.58 

82.16 

10.44 

79.54 

17.24 

Deployment  since 

commissioning  (months) 

9.39 

4.45 

9.05 

4.02 

8.85 

3.52 

9.06 

4.17 

Age 

35.30 

4.16 

29.50 

4.61 

33.55 

4.57 

32.99 

5.08 

Note.  Thirty-nine  candidates  in  the  sample  did  not  answer  the  accession  option  question;  their  data  are  reflected  in  the  "Total" 
column.  In-Service  =  Candidates  who  were  enlisted  Army  Soldiers  prior  to  OCS;  Enlistment  Option  =  Candidates  who  were 
civilians  with  no  military  service  prior  to  OCS;  Hybrid  =  Candidates  with  service  from  another  military  branch,  with  prior 
military  service,  or  with  experience  from  West  Point  or  ROTC  prior  to  OCS. 
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Table  2.2.  Officer  Candidate  School  (OCS)  Full  Sample  Demographics 


In-Service 
(/?  =  553) 

Enlistment 
Option 
(«  =  521) 

Hybrid 
(77  =  231) 

Total 

(N=  1,344) 

Demographic 

N 

% 

N 

% 

N 

% 

N 

% 

Gender 

Male 

404 

73.1 

406 

77.9 

192 

83.1 

1,010 

75.1 

Female 

124 

22.4 

88 

16.9 

27 

11.7 

240 

17.9 

Missing  Values 

25 

4.5 

27 

5.2 

12 

5.2 

94 

7.0 

Ethnicity 

Hispanic 

95 

17.2 

51 

6.3 

35 

15.2 

183 

13.6 

Non-Hispanic 

428 

77.4 

437 

83.9 

184 

79.7 

1,056 

78.6 

Missing  Values 

30 

5.4 

33 

6.3 

12 

5.2 

105 

7.8 

Race 

White 

276 

49.9 

400 

76.8 

163 

70.6 

841 

62.6 

American  Indian/ Alaskan 

3 

0.5 

0 

0.0 

0 

0.0 

3 

0.2 

Asian 

21 

3.8 

30 

5.8 

10 

4.3 

61 

4.5 

Black/African  American 

193 

34.9 

38 

7.3 

38 

16.5 

273 

20.3 

Pacific  Islander 

0 

0.0 

5 

1.0 

1 

0.4 

6 

0.4 

Multiple 

17 

3.1 

33 

6.3 

6 

2.6 

56 

4.2 

Missing  Values 

43 

7.8 

15 

2.9 

13 

5.6 

104 

7.7 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

Age 

32.96 

4.27 

27.26 

6.19 

31.14 

6.67 

30.37 

6.12 

Time  in  Service  (in  months) 

102.72 

59.15 

n/a 

n/a 

73.86 

40.84 

95.12 

56.33 

Time  Deployed  (in  months) 

14.57 

18.51 

n/a 

n/a 

8.75 

12.38 

13.11 

17.28 

AFQT 

69.34 

20.05 

87.55 

10.65 

80.80 

11.77 

78.65 

17.61 

Note.  Thirty-nine  candidates  in  the  sample  did  not  answer  the  accession  option  question;  their  data  is  reflected  in  the  "Total" 
column.  In-Service  =  Candidates  who  were  Army  Soldiers  prior  to  OCS;  Enlistment  Option  =  Candidates  who  were  civilians 
with  no  military  service  prior  to  OCS;  Hybrid  =  Candidates  with  service  from  another  military  branch,  with  prior  military  service, 
or  with  experience  from  West  Point  or  ROTC  prior  to  OCS.  The  data  in  this  table  were  taken  from  Babin  et  al.,  20 1 1 . 


Administrative  Data  Collections 

Although  the  In-Unit  Survey  was  designed  to  capture  a  broad  range  of  verifiable 
performance  and  continuance  information  from  the  2008  OCS  cohort,  we  also  augmented  these 
data  with  information  from  officers’  personnel  records.  The  Army  and  the  Defense  Manpower 
Data  Center  (DMDC)  maintain  a  number  of  databases  with  relevant  personnel  records,  each  with 
their  own  strengths  and  weaknesses.  We  extracted  performance  and  separation  data  primarily 
from  two  sources  -  the  Army  Training  Requirements  and  Resources  System  (ATRRS)  and 
Officer  Master  File  (OMF)  -  and,  where  appropriate,  addressed  weaknesses  in  those  sources 
with  information  from  other  personnel  databases. 

We  first  examined  officer  performance  and  separation  during  Initial  Officer  Training 
(IOT)  -  which  includes  OCS,  BOLC  II,  and  BOLC  III  (or,  in  some  instances,  BOLC  B)  -  by 
extracting  administrative  information  from  the  ATRRS  database.  ATRRS  is  used  to  manage  user 
training  requirements  and  track  scheduling.  The  ATRRS  database  includes  a  record  for  every 
course  an  officer  has  taken  throughout  their  Army  career,  though  we  only  examined  records 
pertaining  to  IOT  courses.  The  database  includes  basic  information  on  whether  the  officer 
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graduated  or  not  from  the  course,  whether  he  or  she  recycled  (i.e.,  had  to  repeat)  a  portion  of  the 
course,  and  the  reasons  for  a  recycle.  We  were  able  to  obtain  BOLC  records  for  most  of  the  2008 
OCS  sample  from  ATRRS  (1,174  out  of  1,387,  or  84.6%,  of  the  records  in  the  database  had 
either  BOLC  II/III  or  BOLC  B  data  in  ATRRS),  but  only  573  (41.3%)  records  from  OCS.  In 
other  words,  more  than  half  the  sample  was  missing  an  ATRRS  record  from  OCS.  As  Table  2.3 
shows,  the  non-graduation  and  recycle  base  rates  for  the  officers  in  our  sample  was  generally 
low,  suggesting  these  metrics  of  performance  and  separation  would  be  difficult  to  predict  with 
the  experimental  predictors.  If  data  were  available  for  a  particular  officer  from  any  one  IOT 
course,  his/her  data  were  included  in  one  of  two  summary  variables  (see  bottom  of  Table  2.3). 
We  used  the  constructed  variables  with  the  highest  base  rate  for  the  largest  sample  (i.e., 
“Recycled  at  Least  Once  During  IOT”  and  “Graduated  from  IOT”)  in  subsequent  analyses. 


Table  2.  S.  Summary  of  Potential  Criterion  Variables  Created  Using  ATRRS 


Variables 

Number 
with  records 

Number 

failing/recycling 

Percentage 

failing/recycling 

Graduation  Variables 

OCS 

573 

41 

7.2% 

BOLC  II 

1,218 

3 

0.2% 

BOLC  III 

1,057 

19 

1.8% 

BOLC  B 

158 

7 

4.4% 

Recycle  Variables 

OCS 

573 

77 

13.4% 

BOLC  II 

1,218 

6 

0.5% 

BOLC  III 

1,057 

47 

4.4% 

BOLC  B 

158 

3 

1.9% 

Summary’  Variables 

Graduated  from  IOT 

1,233 

44 

3.6% 

Recycled  at  Least  Once  During  IOTa 

1,171 

66 

5.6% 

Note.  N=  1,387.  a“Recycled  at  Least  Once  During  IOT”  includes  officers  or  officer  candidates  that  failed  a  course 
or  recycled  at  least  once,  but  completed  the  course.  IOT  =  Initial  Officer  Training. 


Second,  we  extracted  in-unit  officer  performance  and  separation  information  from  the 
OMF,  which  includes  key  infonnation  on  officer  accessions,  promotions,  assignments,  and 
changes  in  status.  We  completed  the  extraction  in  May  of  2012 — after  the  entire  OCS  2008 
sample  had  reached  the  end  of  their  ADSO — to  ensure  the  data  were  mature  enough  for 
interpretation.  In-unit  performance  was  determined  by  examining  the  number  of  medals  each 
officer  in  our  sample  received.  Officers  can  be  awarded  medals  for  many  reasons,  such  as 
deploying  (e.g.,  Afghanistan  Campaign  Medal),  serving  on  active  duty  (e.g.,  National  Defense 
Service  Medal),  and  meritorious  service  (e.g.,  Army  Commendation  Medal).  Only  medals  for 
meritorious  service  and  valor  “counted”  towards  the  awards  variables  (Oliver,  Ingerick,  Khanna, 
Waugh,  &  Wasko,  2012).  We  then  constructed  the  awards  variable  in  two  ways.  First,  we  created 
a  variable  that  was  the  simple  sum  of  the  number  of  above  qualifying  medals  received.  Second, 
recognizing  that  some  medals  are  more  difficult  to  achieve  than  others,  we  created  a  second 
variable  that  differentially  weighted  the  qualifying  awards.  We  derived  the  weights  from  the 
Promotion  Points  Worksheet  for  Non-Commissioned  Officers  (Army  Regulation  600-8-19).  The 
awards,  along  with  their  associated  weights,  can  be  found  in  Table  2.4  (for  more  information 
about  these  medals,  consult  Army  Regulation  600-8-22). 
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Table  2.4.  Valor  and  Merit  Awards  with  Associated  Weights 


Valor 

Merit 

Award 

Weight 

Award 

Weight 

Distinguished  Service  Cross 

35 

Meritorious  Service  Medal 

25 

Silver  Star 

35 

Defense  Meritorious  Service  Medal 

25 

Medal  of  Flonor 

35 

Army  Commendation  Medal 

30 

Distinguished  Flying  Cross 

Bronze  Star  Medal  with  Combat 

35 

Distinguished  Service  Medal 

35 

Distinguishing  Device  V 

Army  Commendation  Medal  with 

30 

Defense  Distinguished  Service  Medal 

35 

Combat  Distinguishing  Device  V 

Joint  Service  Commendation  Medal 

20 

Joint  Service  Achievement  Medal 

15 

with  Combat  Distinguishing  Device  V 

20 

Joint  Service  Commendation  Medal 

20 

Legion  of  Merit 

35 

Army  Achievement  Medal 

15 

Defense  Superior  Service  Medal 

35 

Bronze  Star  Medal 

30 

The  resulting  descriptives  for  the  total  and  weighted  awards  variables  for  the  in-service, 
enlistment  option,  and  hybrid  candidates  can  be  found  in  Table  2.5.  Officers  in  the  in-service 
sample  had  significantly  more  qualifying  awards  than  their  enlistment  option  counterparts  (Raw 
Total  Awards  d  =  1 .54;  Raw  Weighted  Awards  d=  1.31).  One  reason  for  this  difference  is  that 
the  OMF  award  fields  reflect  awards  received  during  enlisted  and  commissioned  service.  Given 
the  confounding  effect  of  time  in  service,  the  awards  variables  were  not  analyzed  further  in  the 
in-service  or  hybrid  samples.  Another  potentially  confounding  influence  on  these  awards 
variables  is  branch  assignment.  Certain  branches,  particularly  those  that  are  more  combat- 
oriented  (e.g.,  Infantry),  are  more  likely  to  receive  medals  than  branches  in  more  support-related 
positions  (e.g.,  Signal  Corps  and  Finance  Corps).  Recognizing  that  there  are  multiple  ways  to 
classify  Army  branches  (see  Anny  Regulation  600-3),  for  the  purposes  of  the  validation  analyses 
we  standardized  the  two  “raw”  awards  variables  by  three  traditional  branch  categories:  (a) 
Combat  Arms,  (b)  Combat  Support,  and  (c)  Combat  Service  Support.7 


Table  2.5.  Award  Descriptives  by  Accession  Option 


In-Service 

Enlistment 

Hybrid 

n 

Mean 

Std. 

n 

Mean 

Std. 

n 

Mean 

Std. 

Raw  Total  Awards 

500 

2.61 

0.96 

427 

1.31 

0.71 

179 

1.50 

0.84 

Raw  Weighted  Awards 

500 

51.55 

22.20 

427 

26.73 

15.13 

179 

29.25 

17.87 

Std.  Total  Awards 

n/a 

n/a 

n/a 

427 

-0.58 

0.68 

n/a 

n/a 

n/a 

Std.  Weighted  Awards 

n/a 

n/a 

n/a 

427 

-0.51 

0.67 

n/a 

n/a 

n/a 

Note.  “Std.”  =  Standardized;  standardization  was  completed  by  branch  category  (Combat  Anns,  Combat  Support,  Combat 
Services  Support). 


7  In  recent  years,  these  categories  have  been  renamed  Maneuver  Fires  and  Effects  Division,  Operational  Support 
Division,  and  Force  Sustainment  Division,  respectively. 
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We  also  constructed  an  additional  variable  examining  officer  separation  primarily  from 
OMF  data,  which  contains  fields  for  separation  dates  and  separation  program  designators 
(SPDs).  SPD  codes  explain  the  reasons  someone  separates  from  active  duty  service.  However, 
individuals  who  separated  prior  to  January  2012  were  not  included  in  the  OMF  extraction.  To 
account  for  this,  we  extracted  administrative  data  from  other  sources  to  fill  in  the  gaps.  From 
these  data,  we  sorted  each  individual  with  a  valid  record  into  one  of  five  groups: 

1 .  Attrition  -  Officer  separated  from  the  Army  prior  to  completing  their  ADSO  for  moral 
(e.g.,  violation  of  Uniformed  Code  of  Military  Justice),  performance  (e.g.,  not  meeting 
fitness  standards),  or  other  pejorative  reasons. 

2.  Injury  -  Officer  separated  from  the  Army  prior  to  completing  their  ADSO  for  a  non- 
pejorative  reason,  usually  an  injury  sustained  while  on  active  duty. 

3.  Did  Not  Commission  -  Records  indicate  that  the  individual  never  received  their 
commission,  but  remains  in  the  Anny  as  an  enlisted  soldier. 

4.  End  of  ADSO  -  Officer  separated  from  the  active  duty  service  after  completing  their  3- 
year  ADSO. 

5.  Anny  Guard  or  Reserve  -  Some  individuals  in  our  sample  attended  OCS,  but  were 
sponsored  by  a  National  Guard  unit  or  the  U.S.  Army  Reserve.  These  individuals 
received  a  commission,  but  did  not  incur  a  3-year  ADSO. 

6.  Current  Active  Duty  Officer  -  Officer  remains  on  active  duty  at  the  time  of  the  data 
extraction. 

We  report  the  percentage  of  officers  in  each  category  in  Table  2.6.  At  the  time  of  our  data 
extraction,  most  of  the  officers  in  our  sample  (over  80%)  were  still  on  active  duty.  For 
subsequent  analyses,  we  treated  the  “attrition,”  “did  not  commission,”  and  “end  of  ADSO” 
groups  as  “separations”  and  individuals  still  on  active  duty  as  “stayers.”  The  remaining  groups 
were  treated  as  missing  data  for  subsequent  analyses.  The  base  rates  for  the  in-service,  enlistment 
option,  and  hybrid  groups  on  this  analysis  variable  can  also  be  found  in  Table  2.6. 

Ongoing  Data  Collections 

Since  the  early  conceptualization  of  this  project,  the  ARJ/HumRRO  research  team 
recognized  the  potential  value  of  collecting  ratings  of  in-unit  performance  from  the  2008  OCS 
cohort’s  supervisors.  Such  ratings  could  expand  and  enrich  the  relatively  limited  longer-tenn 
perfonnance  outcome  information  that  was  available  for  these  officers.  To  the  extent  feasible, 
the  goal  was  to  collect  these  measures  for  use  as  additional  criteria  for  validating  the  OBEF  as 


s  Individuals  with  separation  dates  and  codes  in  the  OMF  were  treated  as  separated.  If  an  individual  was  missing 
OMF  data,  SPD  codes  extracted  from  the  Defense  Manpower  Data  Center  (DMDC)  were  used  instead.  Some 
individuals  were  missing  key  information  for  both  their  OMF  and  DMDC  records  (or  we  could  not  locate  a  record). 
In  these  cases,  we  examined  administrative  records  from  Human  Resources  Command  (HRC)  and  ATRRS  to 
determine  if  (a)  s/he  ever  received  a  commission  as  an  active  duty  officer,  (b)  s/he  was  in  a  U.S.  Army  Reserve  or  an 
Army  National  Guard  unit  when  attending  OCS,  or  (c)  had  attrited  due  to  a  disability  or  was  killed  in  action. 
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well  as  to  provide  additional  convergent  and  discriminant  validity  evidence  for  other  criterion 
measures  being  examined.  The  effort  to  collect  supervisor  ratings  was  particularly  challenging 
because  the  officers  in  the  2008  OCS  cohort  were  spread  out  in  locations  throughout  the  world. 
Collecting  such  data  also  required  compliance  with  the  human  subjects  protection  guidelines  and 
review  process  (i.e.,  ARI’s  Institutional  Review  Board;  IRB).  This  increased  the  complexity  of 
the  data  collection  procedures  which  needed  to  be  developed,  and  made  it  more  difficult  to 
obtain  high  response  rates.  In  spite  of  these  challenges,  a  semi-automated  web-based  approach 
for  collecting  supervisor  ratings  was  successfully  developed  and  field  tested.  This  approach  and 
the  rating  scale  instrument  used  are  described  in  more  detail  in  Appendix  C.  The  collection  of  in¬ 
unit  perfonnance  ratings  was  ongoing  at  the  time  of  this  writing;  hence,  the  ratings  data  are  not 
discussed  further  in  the  main  body  of  this  report. 


Table  2.6.  Separation  Status  by  Accession  Option 


Separation  Status 

In-Service 
(n  =  565) 

Enlistment 
(n  =  529) 

Hybrid 
(n  =  237) 

Total 

(n  =  1,391) 

Current  Active  Duty  Officer 

89.20% 

77.32% 

74.68% 

80.59% 

Separated:  Attrition 

1.59% 

4.16% 

0.84% 

2.44% 

Separated:  Injury 

0.18% 

1.13% 

0.42% 

0.65% 

Separated:  Did  not  commission 

1.42% 

1.32% 

4.64% 

1.87% 

Separated:  End  of  ADSO 

3.19% 

10.02% 

5.06% 

6.11% 

Army  Guard  or  Reserve  at  OCS 

2.65% 

4.35% 

11.81% 

4.89% 

Missing 

1.77% 

1.70% 

2.53% 

3.45% 

Analysis  Variable 

In-Service 

Enlistment 

Hybrid 

Total 

Current  Active  Duty  Officer 

89.20% 

77.32% 

74.68% 

80.59% 

Separations:  Attrition,  Did  not 
commission,  or  End  of  ADSO 

6.19% 

15.50% 

10.55% 

10.42% 

Missing  or  Other 

4.60% 

7.18% 

14.80% 

8.99% 

Note.  The  “Analyses  Variable”  was  used  in  subsequent  analyses.  Missing  or  Other  =  Missing,  separated  due  to  injury,  or  was  in 
the  Guard  or  Reserve  when  attending  OCS. 


In-Unit  Survey  Scoring 

The  In-Unit  Survey  includes  a  number  of  items  related  to  officer  performance  and 
continuance.  To  narrow  these  items  into  a  usable  set  of  scores  representing  the  criterion  space, 
we  consulted  a  recent  review  of  “individual  performance,”  defined  by  the  actions  one  takes  on 
the  job,  in  the  industrial  and  organizational  psychology  literature  (Campbell,  2012).  Campbell 
provides  an  integrative  eight-factor  perfonnance  model.  We  revised  some  the  factors  to  be  more 
Anny  specific  and  consistent  with  earlier  taxonomies  (Campbell  &  Knapp,  2001;  Campbell, 
McCloy,  Oppler,  &  Knapp,  1993)  and  added  a  couple  of  dimensions  to  more  specifically  include 
continuance  intentions  and  attitudes,  which  are  of  particular  interest  to  the  Anny.  The  following 
12  factors  guided  subsequent  analyses: 

1 .  Branch-specific  technical  proficiency  -  “Branches”  are  Army  job  categories,  such  as 
Infantry,  Annor,  Aviation,  and  so  forth.  In  the  Army,  officers  are  expected  to  be 
“generalists,”  but  are  also  trained  in  branch-specific  duties.  This  factor  refers  to  officer 
perfonnance  of  these  branch-specific  duties. 
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2.  Army-wide  technical  proficiency  -  In  the  Anny,  there  are  a  number  of  technical 
core/warrior  tasks  that  everyone  must  be  able  to  perfonn  (e.g.,  navigation,  weapons). 

This  factor  refers  to  officer  performance  of  these  common  tasks. 

3.  Communication  proficiency  -  Oral  and  written  communication  are  critical  components  of 
officer  jobs.  This  factor  reflects  this  proficiency,  not  the  correctness  of  what  is  being 
communicated. 

4.  Demonstrating  effort  -  This  factor  reflects  the  effort  exhibited  by  the  officers  on  a 
consistent  basis  and  particularly  under  adverse  (e.g.,  dangerous)  conditions. 

5.  Personal  discipline  -  This  factor  reflects  actions  officers  take  that  can  have  a  negative 
effect  on  Army  goals.  It  is  most  similar  to  the  “counterproductive  work  behavior” 
dimension  described  in  Campbell  (2012). 

6.  Physical  fitness  -  This  factor  refers  to  the  extent  to  which  officers  meet  or  exceed  the 
Anny’s  standards  for  fitness,  strength,  and  weight. 

7.  Leadership  -  The  leadership  factor  includes  interpersonal  aspects  of  officer  behaviors 
relevant  to  their  supervisory  role.  Specific  behaviors  include  initiating  structure, 
coaching,  and  serving  as  a  model. 

8.  Management  -  Unlike  the  leadership  factor,  the  management  factor  emphasizes  more 
administrative  behaviors  related  to  officers’  roles.  Specific  behaviors  include  goal  setting, 
coordinating,  and  decision  making. 

9.  Career  Intention  -  The  ninth  factor  refers  to  the  extent  to  which  the  officer  intends  to 
make  the  Anny  a  long-tenn  career. 

10.  Branch  Satisfaction  -  This  factor  is  intended  to  assess  the  extent  to  which  the  officer  is 
satisfied  with  their  branch  assignment,  a  factor  qualitatively  thought  to  be  related  to 
officer  satisfaction  and  continuance  (Oliver,  Ardison,  Russell,  &  Babin,  2011). 

11.  Commitment  to  the  Army  -  This  factor  reflects  an  officer’s  level  of  commitment  to  the 
Anny,  which  has  been  linked  to  retention  in  enlisted  populations  (Strickland,  2005).  For 
this  reason,  it  is  frequently  included  in  Army  validation  analyses  and  is  included  here  for 
completeness. 

12.  Morale  -  The  morale  factor  assesses  an  officer’s  level  of  morale  at  the  current  moment. 
As  with  the  Branch  Satisfaction  and  Commitment  to  the  Army  factors,  this  factor  is  also 
thought  to  be  predictive  of  continuance.  Morale  was  assessed  at  three  data  collection  time 
points. 

As  described  earlier  in  this  chapter,  the  In-Unit  Survey  was  designed  to  assess  officers’ 
attitudes,  retention  intentions,  and  perfonnance  both  in  training  and  in-unit.  However,  the  items 
were  not  designed  specifically  to  tap  all  of  the  perfonnance/retention  dimensions  listed  above. 
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To  map  the  items  in  the  In-Unit  Survey  to  the  factors  above,  5  independent  raters  were  recruited 
and  asked  to  provide  2  judgments  regarding  each  item/factor  combination.  First,  raters  were 
asked  to  determine  whether  the  item  was  a  direct  measure  of  the  factor,  a  non-direct  measure  of 
the  factor,  or  not  related  to  the  factor.  Second,  the  raters  were  asked  to  provide  the  directionality 
of  the  linkage  (e.g.,  a  positive  or  negative  indicator  of  the  factor).  An  item  was  linked  to  a  factor 
when  3  of  the  5  raters  identified  a  direct  linkage.  Variables  directly  linked  to  a  factor  by  2  raters 
and  indirectly  linked  by  at  least  one  additional  rater  were  also  considered  to  represent  the 
criterion  factor. 

Descriptive  statistics  and  intercorrelations  among  the  linked  variables  within  a  criterion 
factor  were  considered  in  fonnulating  the  final  variables.  Qualitative  information  was  also 
considered.  For  example,  even  though  raters  linked  reception  of  the  “Expert  Field  Medical”  and 
the  “Expert  Infantry”  badges  to  branch-specific  technical  performance,  much  of  the  sample  was 
ineligible  for  these  badges,  and,  hence,  we  did  not  include  them  in  the  branch-specific  technical 
performance  composite.  Also,  wherever  possible,  we  relied  on  objective  rather  than  subjective 
metrics  for  each  factor  because  objective  metrics  are  thought  to  be  more  accurate  in  a  self-report 
context  (Knapp,  Campbell,  Bonnan,  Pulakos,  &  Hanson,  2001).  The  results  of  the  linkage  and 
analysis  activities  are  summarized  in  Table  2.7. 

As  Table  2.7  demonstrates,  differential  item  scaling  (e.g.,  ordinal,  dichotomous)  led  to 
unique  combinations  of  items.  For  example,  some  items  combined  an  ordinal  scale  (e.g.,  self- 
ratings  of  technical  competence)  with  one  or  more  dichotomous  items  (e.g.,  “Reason  for  restart 
BOLC  III/B  -  Academic  failure”).  However,  the  base  rate  for  some  of  these  dichotomous  items 
was  low  (e.g.,  only  13  out  of  543  officers  indicated  a  BOLC  III/B  Academic  failure).  To  account 
for  this,  items  with  a  low  base  rate  were  sometimes  treated  as  “bonus”  adjustments  to  the  ratings. 
For  example,  Branch  Satisfaction  was  based  on  an  officers’  self-reported  rating  of  satisfaction 
with  their  branch  assignment,  but  adjusted  downward  by  one  point  if  they  indicated  leaving 
Active  Duty  Service  because  they  could  not  get  the  branch  they  wanted.  This  procedure  allowed 
us  to  include  nearly  all  variables  linked  to  a  construct  in  our  analysis,  even  the  variables  with  a 
low  base  rate.  Decisions  such  as  these  were  made  for  all  of  the  criterion  factors,  as  described  in 
Table  2.7. 

In  subsequent  analyses  reported  in  Chapter  3,  the  factors  described  in  Table  2.7  were 
used  as  the  core  criteria  to  evaluate  whether  the  OBEF  measure  predicts  in-unit  officer 
performance,  retention,  and  attitudes.  To  this  list,  we  added  the  key  administrative  criteria  from 
Tables  2.3,  2.5,  and  2.6.  The  descriptive  statistics  and  intercorrelations  for  the  criterion  variables 
used  in  subsequent  chapters  can  be  found  in  Table  2.8. 

Summary 

The  purpose  of  this  chapter  was  to  describe  the  research  methodology.  Recall  that  one  of 
the  main  purposes  of  the  present  research  is  to  validate  the  OBEF  against  in-unit  criteria.  All  of 
the  procedures  feed  into  the  criterion  variables  used  to  accomplish  this  objective.  The  pattern  of 
intercorrelations  provides  convergent  validity  evidence  for  the  criterion  measures,  suggesting 
they  are  usable  for  validation  purposes.  Future  data  collections  will  hopefully  “fill  the  gaps”  in 
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the  performance  space,  such  as  the  “Demonstrating  effort”  factor,  not  captured  in  the  current 
analysis,  and  collect  more  mature  administrative  criteria  than  what  was  achieved  here. 
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Table  2.7.  Summary  of  Linkage  Results 


Criterion  Factors 

Final  Items  Linked 

Scoring  Approach 

Branch-Specific 

Technical 

Proficiency 

Item  41:  Technical  Competence  Isubiectivel 

Item  12:  Did  you  graduate  from  BOLC  III/B? 

Item  1  lc:  Reason  for  restart  BOLC  III/B  -Excessive 
number  of  retests 

Item  11a:  Reason  for  restart  BOLC  III/B  -  Academic 
failure 

Start  with  Technical  Competence  rating  and 
subtract  one  point  if  “yes”  was  indicated  for 
any  of  the  remaining  items. 

Army-Wide 

Technical 

Proficiency 

Item  42:  Tactical  Competence  (subjective) 

Item  15:  Weapons  Qualification  Score 

Start  with  Tactical  Competence  rating  Add  1 
point  to  tactical  competence  rating  if  officer 
reports  earning  “Marksman”  on  his/her  last 
weapons  qualification  score  and  subtract  1 
point  if  officer  reports  they  did  not  qualify. 

Communication 

Proficiency 

NO  ITEMS  LINKED 

Demonstrating 

Effort 

NO  ITEMS  LINKED 

Personal  Discipline 

Item  9g:  Reason  for  restart  OCS  -  Disciplinary  reasons 

Item  26:  Have  you  received  a  General  officer  Letter  or 
Memorandum  of  Reprimand? 

Item  29:  Have  vou  received  formal  counseling  about 
your  behavior  or  discipline? 

All  items  are  dichotomous,  yes/no.  If  “yes”  is 
marked  for  any  of  the  three  items,  assign  a 
value  of  1 .  If  “no”  is  marked  for  all  three 
items,  assign  a  value  of  zero. 

Physical  Fitness 

Item  39:  Physical  Fitness.  Health  and  Endurance 
(subjective) 

Item  38:  LAST  APFT  Score 

Standardized  each  variable  and  average  to 
produce  a  composite  score. 

Leadership 

Item  34:  Did  vou  receive  the  following  on  your  most 
recent  OER:  "Outstanding  Perfonnance  Must 

Promote”? 

Item  32:  Has  a  senior  officer  recommended  vou  for  a 
command  on  the  OER? 

All  items  are  dichotomous,  yes/no.  If  “yes”  is 
marked  for  both  items,  assign  a  value  of  1 .  If 
“no”  is  marked  for  either  item,  assign  a  value 
of  zero. 

Management 

NO  ITEMS  LINKED 

Career  Continuance 
Intentions 

Item  59:  Current  active  duty  career  intentions 

Item  60:  Have  vou  submitted  a  request  release  from 
Active  Duty  (REFRAD)? 

If  officer  indicates  s/he  is  “probably”  or 
“definitely”  planning  to  leave  the  Army  in 

Item  59,  or  says  they  plan  to  submit  a 

REFRAD  (whether  they  have  done  so  or  not; 
Item  60),  assign  a  value  of  0  (plans  to 
separate).  If  the  officer  indicates  no  intention 
to  submit  a  REFRAD  or  indicates  an 
intention  to  stay  beyond  their  ADSO,  assign 
a  value  of  1  (plans  to  stay  in  the  Army). 

Branch  Satisfaction 

Item  20:  How  satisfied  are  vou  with  vour  branch 
assignment? 

Item  61k:  Reason  for  leaving  Active  Duty  Service  - 
Could  not  get  the  branch  I  wanted 

Start  with  branch  satisfaction  item  and 
subtract  1  point  if  person  responds  “yes”  to 
item  61k. 

Commitment  to  the 
Army 

Items  46-49:  Annv  Affective  Commitment  items 

Take  the  arithmetic  average  of  the 
component  items  for  the  Affective 

Commitment  scale,  reverse  coding  as 
necessary. 

Morale 

Item  45:  How  would  vou  rate  vour  current  level  of 
morale? 

Note.  Survey  items  can  be  found  in  Appendix  B. 
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Table  2.8.  Criterion  Variable  Descriptive  Statistics  and  Intercorrelations 


K> 

4^ 


n  M 


Continuously-Scaled  Criteria 


1. 

Army  Commitment a 

501 

3.87 

2. 

Branch-Specific  TK  a 

513 

4.36 

3. 

Army- Wide  TK  a 

509 

4.45 

4. 

Physical  Fitness  a 

494 

-0.03 

5. 

Std.  Total  Awards'3 

458 

-0.53 

6. 

Std.  Weighted  Awards'3 

458 

-0.46 

7. 

Branch  Satisfaction  a 

501 

3.78 

8. 

Morale  a 

501 

3.52 

Dichotomously-Scaled  Criteria 

9. 

Career  Continuance  Intent. a 

543 

0.84 

10. 

Active  Duty  Separation b 

1,266 

0.11 

11. 

Leadership  a 

543 

0.76 

12. 

IOT  Recycles  b 

1,171 

0.06 

13. 

IOT  Graduation  b 

1,233 

0.96 

14. 

Personal  Discipline  a 

543 

0.04 

SD 

1 

2 

3 

4 

5 

0.97 

.92 

0.83 

.13 

- 

0.86 

.18 

.53 

- 

0.94 

.13 

.17 

.22 

.75 

0.73 

.26 

.11 

.08 

.19 

- 

0.73 

.26 

.19 

.11 

.21 

.93 

1.52 

.26 

.17 

.08 

-.04 

.13 

1.03 

.61 

.12 

.20 

.21 

.19 

0.37 

.43 

.05 

.16 

.04 

.16 

0.32 

-.14 

-.14 

-.15 

-.01 

-.04 

0.43 

.12 

.21 

.19 

.10 

.21 

0.23 

.06 

-.14 

-.11 

.02 

.02 

0.19 

-.04 

.02 

-.01 

-.04 

.02 

0.20 

-.04 

.03 

-.10 

-.13 

-.22 

6  7  8  9  10  11  12 


.16 

.14  .38 


.11 

.14 

.43 

- 

,04 

-.03 

-.13 

-.22 

- 

.21 

.05 

.03 

.03 

-.02 

- 

.03 

-.12 

.04 

.06 

.16 

-.08 

- 

.03 

.02 

-.05 

-.07 

-.39 

.12 

-.51 

.20 

-.03 

-.11 

-.06 

-.05 

-.12 

-.01 

13 


.03 


Note.  Correlations  performed  on  the  raw  data.  Where  appropriate,  coefficient  alphas  are  along  the  diagonal  in  italics.  TK  =  Technical  Knowledge.  Std.  =  Standardized.  Intent.  = 
Intentions.  IOT  =  Initial  Officer  Training.  The  “Awards”  results  are  limited  to  the  enlistment  option  sample. 

“Refer  to  Table  2.7  for  scaling  infonnation. 

b  Refer  to  “Administrative  Data  Collections”  section  for  infonnation  about  these  variables.  The  following  scaling  applies:  IOT  Recycles  (0  =  no  recycles,  1  =  at  least  one 
recycle),  IOT  Graduation  (0  =  did  not  graduate,  1  =  graduated),  and  Active  Duty  Separation  (1  =  separated  and  0  =  current  active  duty  officer). 


CHAPTER  3:  PREDICTION  OF  OFFICER  PERFORMANCE  AND  CONTINUANCE 

WITH  THE  OBEF 


Bethany  H.  Bynum 

Human  Resources  Research  Organization 

This  chapter  describes  the  analyses  conducted  to  validate  the  Officer  Background  and 
Experiences  Fonn  (OBEF)  as  a  predictor  of  OCS-commissioned  officer  performance, 
continuance,  and  attitudes  in-unit  as  officers  near  the  end  of  their  active  duty  service  obligation 
(ADSO) — 3  years  after  receiving  their  commission.  As  described  in  Chapter  2,  three  OBEF 
predictors  were  examined:  (a)  the  Core  Rational  Biodata  Inventory  (RBI)  scales,  (b)  the  Core 
Work  Values  scales,  and  (c)  the  Situational  Judgment  Test  (SJT).  The  perfonnance  composites 
developed  in  the  first  phase  of  research  conducted  in  2008-2009  also  were  examined  for  their 
predictive  efficacy  beyond  OCS  (Allen,  2011).  The  primary  objectives  of  this  chapter  were  to  (a) 
detennine  which  OBEF  scales  best  predict  important  outcomes,  and  (b)  validate  the  composites 
constructed  previously  using  longer-term  criteria  collected  well  beyond  the  initial  OCS  training 
period.  This  validation  effort  focused  on  the  enlistment  option  and  in-service  samples.  The 
hybrid  sample  was  excluded  from  subsequent  analysis  due  to  their  small  sample  size  and 
heterogeneity  in  previous  military  experience. 

Background 

In  the  first  phase  of  the  OCS  research  (Russell  &  Tremble,  2011),  four  versions  of  the 
OBEF  were  administered.  The  four  versions  included  both  “core”  and  “experimental”  content. 
The  “core”  content  generally  comprised  of  scales  and  constructs  that  previous  research  had 
suggested  would  be  predictive  of  officer  perfonnance  and  retention.  The  “experimental”  content 
consisted  of  new  scales,  some  of  which  were  included  mainly  for  research  purposes.  In  the 
context  of  the  current  research,  this  distinction  is  important  because  the  “core”  scales  were 
generally  administered  in  more  versions  of  the  OBEF  than  the  experimental  scales.  Therefore, 
the  two  “core”  sets  of  OBEF  scales  (Work  Values  and  RBI)  administered  in  the  first  phase  of 
OCS  research  were  of  primary  interest  in  the  present  research. 

Another  reason  the  Core  RBI  and  Core  Work  Values  scales  were  of  primary  interest  was 
their  efficacy  for  predicting  key  outcomes  of  interest  in  OCS.  In  general,  the  component  scales 
for  the  Core  RBI  and  Core  Work  Values  measures  had  higher  validity  coefficients  than  the 
component  scales  for  other  measures  included  across  versions  of  the  OBEF  for  predicting  OCS 
perfonnance,  affective  commitment,  and  Regular  Anny  career  intentions  (Allen,  2011).  This  was 
true  for  both  the  in-service  and  enlistment  option  samples.  For  this  reason,  the  Core  RBI  and 
Core  Work  Values  scales  were  used  to  fonnulate  two  predictor  composites — an  in-service  and 
enlistment  option  version.  The  composites  exhibited  acceptable  reliability,  little  skew,  and  few 
subgroup  differences.  Both  composites  significantly  predicted  affective  commitment,  career 
intentions,  OCS  leadership  scores,  OCS  fitness  scores,  and  the  total  OCS  score. 

In  addition  to  the  Core  RBI  and  Work  Values  scales,  one  additional  “experimental”  OBEF 
measure  was  included  in  the  present  analysis — the  Situational  Judgment  Test  (SJT).  We  included  the 
SJT  in  this  analysis  for  three  reasons.  First,  the  SJT  was  administered  to  a  higher  proportion  of  OCS 
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candidates  than  most  of  the  other  experimental  scales.  This  suggests  that  missing  data  will  be  less  of 
an  issue  with  the  SJT  than  with  other  experimental  OBEF  scales.  Second,  meta-analytic  evidence 
suggests  that  SJTs  have  incremental  validity  for  predicting  job  perfonnance  beyond  both  cognitive 
ability  and  personality  measures  (McDaniel,  Hartman,  Whetzel,  &  Grubb,  2007).  While  we  did  not 
find  strong  evidence  that  the  SJT  predicted  key  perfonnance  and  retention  outcomes  at  the  End  of 
OCS  (Allen,  2011),  this  may  have  partially  been  due  to  its  narrow  content  focus.  As  described  in 
Chapter  2,  the  OCS  SJT  was  designed  to  evaluate  candidate  judgment  in  leadership  situations. 
Candidates  in  OCS  may  have  fewer  opportunities  to  demonstrate  their  leadership  judgment  in  a 
training  enviromnent  than  in-unit.  Therefore,  the  SJT  may  have  more  opportunity  to  predict  key 
officer  outcomes  after  they  join  their  units  than  when  they  were  in  training.  Third,  SJTs  are  thought 
be  less  fakable  than  self-report  measures  (Hooper,  Cullen,  &  Sackett,  2006).  This  suggests  the  SJT 
would  be  a  promising  instrument  for  operational  applications,  provided  there  is  evidence  it  predicts 
key  perfonnance  and  retention  outcomes. 

In  summary,  the  Core  RBI,  Core  Work  Values,  and  experimental  SJT  scales  from  the 
original  OBEF  were  examined  in  the  present  analysis. 

Analytic  Approach 

In  the  initial  research,  the  potential  for  each  OBEF  measure  to  contribute  to  the  Army’s 
current  selection  procedures  was  determined  by  examining  the  incremental  validity  of  each 
measure  beyond  a  baseline  measure,  in  this  case,  the  Anned  Forces  Qualification  Test  (AFQT; 
Allen,  2011).  However,  the  current  dataset  presented  a  number  of  unique  challenges  that  required 
modification  to  this  approach.  First,  as  is  often  the  case  with  longitudinal  studies  wherein 
multiple  instruments  are  administered  over  time,  there  is  a  large  amount  of  missing  data  in  this 
research.  Second,  shown  in  Table  2.1,  the  variance  in  the  AFQT  baseline  predictor  is  quite  low, 
which  may  downwardly  bias  its  predictive  efficacy  in  a  broader  sample.  Third,  there  is  a  large 
number  of  predictors  in  some  of  the  measures  relative  to  the  number  of  cases  in  our  small 
sample.  This  may  inflate  the  predictive  efficacy  of  the  OBEF  measures.  To  account  for  these 
factors,  the  following  procedures  were  applied  to  examine  the  incremental  validity  of  the 
experimental  measures: 

1 .  Computed  the  incremental  validity  using  Ordinary  Least  Squares  (OLS)  regression  with 
Full  Information  Maximum  Likelihood  (FIML)  missing  data  analyses. 

2.  Detennined  the  statistical  significance  of  the  incremental  validity  added  by  the  OBEF 
measures  beyond  AFQT. 

3.  Estimated  a  variance/covariance  matrix  using  FIML  missing  data  analyses. 

4.  Corrected  the  variance/covariance  matrix  for  direct  range  restriction  on  AFQT. 

5.  Computed  incremental  validity  estimates  using  the  corrected  variance/covariance  matrix. 

6.  Corrected  the  incremental  validity  estimates  for  shrinkage. 

These  steps  are  described  in  more  detail  below. 
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Missing  Data  Estimation 

Studies  have  shown  that  large  losses  of  data  can  lead  to  low  power  and  downwardly 
biased  estimates  (Enders  &  Bandalos,  2001;  Muthen,  Kaplan,  &  Hollis,  1987;  Roth,  1994; 
Schafer  &  Graham,  2002).  As  is  often  the  case  with  longitudinal  research,  we  were  unable  to 
capture  criterion  data  for  all  of  the  individuals  who  took  the  OBEF  measure  in  2008.  Of  the  over 
1,300  candidates  who  took  the  OBEF,  543  (40.4%)  responded  to  the  In-Unit  Survey. 
Additionally,  because  multiple  versions  of  the  OBEF  were  administered  during  the  Beginning  of 
OCS  data  collection,  there  are  individuals  with  missing  predictor  data — 974  (70.0%)  completed 
the  Core  RBI  scales,  972  (69.9%)  completed  the  Core  Work  Values  scales,  and  718  (51.6%) 
completed  the  SJT.  Combining  these  factors,  using  traditional  methods  to  deal  with  missing  data 
such  as  listwise  deletion  would  have  resulted  in  very  small  samples,  potentially  biased  estimates, 
and  limitations  on  the  interpretability  of  the  results. 

To  deal  with  the  missing  data,  we  opted  to  conduct  regression  analyses  using  FIML 
missing  data  estimation.  Studies  have  shown  that  using  FIML  produces  parameter  estimates  that 
are  less  biased  than  relying  on  multiple  imputation,  listwise,  or  pairwise  deletion  (i.e.,  Enders, 
2001).  FIML  uses  all  of  the  available  data  to  estimate  the  likelihood  value  of  the  parameter 
estimates  for  each  individual.  Specifically,  when  there  are  missing  observations,  matrices  are 
produced  that  incorporate  both  observed  and  missing  parameter  infonnation.  The  information 
associated  with  the  complete  portion  of  the  vector  is  used  to  estimate  the  likelihood  value  of  the 
variables  with  missing  data  (Enders,  2006).  Note  that  unlike  other  missing  data  estimation 
methods,  data  are  not  imputed  into  the  database.  The  missing  data  are  accounted  for  by  including 
additional  infonnation  to  estimate  the  parameters  in  the  analysis. 

Missing  data  analyses  were  conducted  using  the  FIML  feature  in  Mplus  (Muthen  & 
Muthen,  2004).  To  ensure  there  was  adequate  information  to  estimate  parameters,  we  examined 
the  frequency  of  missing  data  on  each  predictor  scale.  Among  the  Core  RBI,  Core  Work  Values 
and  SJT,  there  are  20  scales.  In  all,  43. 1%  of  the  sample  was  missing  data  on  two  or  fewer  of  the 
scales,  53.6%  of  the  sample  was  missing  data  on  between  eight  and  eleven  scales,  and  3.3%  of 
the  sample  was  missing  data  between  17  and  20  of  the  scales.  We  decided  to  limit  the  sample  to 
officers  with  data  on  at  least  12  of  the  20  scales  (60%  of  the  core  predictor  data).  This  effectively 
removed  anyone  who  did  not  complete  the  core  RBI  scales.  Limiting  the  sample  to  individuals 
with  data  on  more  than  12  scales  would  have  effectively  limited  the  sample  to  nearly  the  same 
number  as  listwise  deletion.  The  final  analyses  sample  was  939  (in-service  =  390;  enlistment 
option  =  368;  hybrid  =  166).  We  conducted  most  of  the  analyses  on  the  in-service  and  enlistment 
option  subsamples. 

Incremental  Validity 

To  assess  the  criterion-related  validity  of  the  OBEF,  Ordinary  Least  Squares  (OLS) 
regression  analyses  were  conducted  for  the  continuous  criterion  variables  and  logistic  regression 
analyses  were  conducted  for  the  dichotomous  criterion  variables.  For  each  criterion  variable,  we 
examined  the  incremental  validity  of  each  predictor  set  (i.e.,  RBI,  Work  Values,  and  SJT)  over 
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AFQT.  Incremental  validity  is  an  estimate  of  the  change  in  the  multiple  correlation  (A R)  when  a 
new  predictor  or  set  of  predictors  is  added  to  a  regression  equation.  New  predictors  that  add 
validity  beyond  that  already  afforded  by  AFQT  are  more  likely  to  prove  useful  for  selection 
purposes.  The  analyses  consisted  of  a  two-step  process.  First,  the  criterion  variable  was  regressed 
on  AFQT.  Second,  all  of  the  scales  comprising  the  experimental  predictors  (i.e.,  RBI,  Work 
Values,  or  SJT)  were  added  to  the  regression  equation  in  the  same  step.  The  difference  between 
the  multiple  correlations  ( R )  produced  in  the  AFQT-only  model  and  the  AFQT-plus-predictors 
model  was  then  calculated  to  detennine  incremental  validity.  For  logistic  regression,  a  pseudo-R 
estimate  was  used  in  place  of  the  multiple  correlation. 

Two  criteria  were  used  to  assess  whether  the  OLS  incremental  validity  coefficients  (A R) 
were  statistically  significant:  (a)  the  change  in  the  log-likelihood  ratio  test  and  (b)  the  F-test 
associated  with  the  change  in  R-squared.  Traditional  regression  approaches  rely  on  the  F-test  to 
assess  significance.  However,  the  computation  of  the  R-squared  change  F-test  is  dependent  on 
the  sample  size,  such  that  large  sample  sizes  tend  to  lead  to  more  significant  results.  When  using 
FIML  missing  data  analyses,  there  is  no  single  value  of  N  that  applies  to  the  entire  sample.  Using 
the  full  sample  size  (including  cases  with  incomplete  data)  or  relying  on  the  listwise  sample  size 
is  likely  to  result  in  Type  I  and  Type  II  errors,  respectively.  As  a  result,  when  using  FIML, 
researchers  recommend  relying  on  the  Log-likelihood  to  assess  how  well  each  model  fits  the  data 
(Enders,  2001).  The  likelihood  value  is  an  assessment  of  how  well  the  estimated  parameters  fit 
the  observed  data.  Minus  two  times  the  log  of  the  likelihood  (-2  Log-likelihood  or  -2LL)  is  a  chi- 
square  statistic  and  can  by  modeled  using  the  standard  assumptions  underlying  the  chi-square 
distribution.  The  difference  between  the  -2  Log-likelihood  values  from  the  AFQT  only  model 
and  the  AFQT  plus  predictor  model  was  used  to  detennine  whether  the  models  significantly 
differ,  and  thus  whether  the  addition  of  the  predictor  set  significantly  improved  model  fit. 

To  compare  the  results  of  the  initial  OBEF  validation  (Allen,  201 1)  we  decided  it  would 
be  helpful  to  present  the  F-test  associated  with  the  change  in  R-square.  In  an  effort  to  be 
conservative,  the  listwise  sample  size  (i.e.,  159  for  the  in-service  sample  and  123  for  the 
enlistment  option  sample)  was  used  for  computing  the  F-test  associated  with  change  in  R- 
squared.  Results  of  the  -2  Log-likelihood  test  and  the  F-test  are  presented  for  the  continuous 
criterion  variables.  Since  the  logistic  regression  analyses  rely  on  a  pseudo-R  estimate,  it  is  not 
appropriate  to  compute  an  F-test  associated  with  the  difference  in  the  change  in  R-square. 
Therefore,  the  -2  Log-Likelihood  test  was  used  to  assess  the  incremental  validity  for  the 
dichotomous  criterion  variables. 

Range  Restriction 

As  discussed  in  Chapter  2,  the  range  on  the  AFQT  is  restricted  for  the  current  sample, 
and  particularly  for  the  enlistment  option  sample.  Range  restriction  on  AFQT,  uncorrected,  will 
lead  to  underestimates  of  the  validity  of  AFQT  and,  potentially,  overestimates  of  incremental 
validity.  Therefore,  we  corrected  the  regression  coefficient  between  the  candidates’  AFQT  score 
and  each  criterion  for  direct  range  restriction  using  Lawley’s  formula  (cf.  Hunter  &  Schmidt, 
1990).  Similar  to  analyses  conducted  on  the  2008  cohort,  the  population  AFQT  standard 
deviation  estimates  were  derived  from  the  1997  National  Longitudinal  Survey  (NLSY97) 
(Bureau  of  Labor  Statistics,  2005),  a  study  commissioned  by  the  Department  of  Labor  that 
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includes  data  for  8,984  youths  on  a  variety  of  variables,  including  college  experience  and 
ASVAB  subtest  scores.  The  college  experience  samples  were  used  to  derive  the  population 
standard  deviation  estimates  for  our  samples.  Specifically,  youths  from  the  NLSY97  data  with 
“some  college”  experience  (between  12  and  16  years  of  education)  were  used  as  the  population 
sample  for  the  in-service9  candidates  in  our  sample,  while  youths  from  the  NLSY97  data  who 
were  “college  graduates”  (16  or  more  years  of  schooling)  were  used  as  the  population  sample  for 
the  enlistment  option  candidates  in  our  sample. 

The  linear  regression  analyses  incorporating  the  range  restriction  were  completed  by 
creating  two  correlation  matrices,  one  for  the  in-service  sample  and  one  for  the  enlistment  option 
sample.  The  correlation  matrices  were  estimated  using  FIML  missing  data  analyses.  Note  that 
the  difference  in  estimation  techniques  using  FIML  regression  compared  to  using  standard  OLS 
regression  with  a  FIML  estimated  correlation  matrix  may  lead  to  slightly  discrepant  results. 
Specifically,  FIML  regression  uses  missing  data  information  to  estimate  the  regression 
parameters  whereas  estimating  a  correlation  matrix  with  FIML  relies  on  the  missing  data 
information  to  estimate  the  correlations.  Then,  the  correlation  estimates  are  used  to  directly 
compute  the  regression  parameters. 

We  corrected  all  of  the  AFQT  intercorrelations  for  direct  range  restriction.  Then  OLS 
regression  analyses  were  performed  on  the  corrected  correlation  matrices.  The  change  in 
multiple  R  from  step  one  to  step  two  served  as  the  primary  diagnostic  to  evaluate  each  measure. 

Shrinkage 

Sample-specific  error  could  potentially  inflate  the  estimates  of  R  for  predictor  measures 
with  small  sample  sizes  and  many  scales.  As  a  result,  variations  in  sample  sizes  and  the  number 
of  scales  constituting  each  predictor  measure  make  cross-measure  comparisons  difficult.  To 
address  this  issue,  we  adjusted  the  observed  incremental  validity  estimates  using  Burket’s  (1964) 
formula  for  shrinkage  (cf.  Formula  8;  Schmitt  &  Ployhart,  1999).  Calculating  the  corrected 
incremental  validity  estimates  involved  two  additional  steps: 

1 .  Using  the  corrected  (for  direct  range  restriction)  correlations  among  the  experimental 
predictor,  AFQT,  and  the  selected  criteria,  adjust  the  correlations  between  the 
predictors  and  continuously-scaled  criteria  for  sample  size  and  number  of  predictors 
using  Burket’s  (1964)  formula  for  shrinkage. 

2.  Calculate  the  corrected  incremental  validity  estimates  for  the  experimental  predictors 
by  subtracting  the  shrunken  R  (the  R  from  Step  1)  associated  with  an  AFQT-only 
model  from  the  shrunken  R  obtained  from  the  AFQT  plus  predictor  model. 


9  Although  the  current  policy  specifies  that  everyone,  even  officer  candidates  entering  through  the  In-Service 
Program  must  have  a  college  degree  prior  to  enrolling  in  OCS,  the  current  sample  includes  former  In-Service 
candidates  who  had  not  yet  completed  a  college  degree  when  completing  OCS.  Therefore,  for  the  current  research 
we  relied  on  the  NLSY97  data  of  individuals  with  “some  college”  experience  to  derive  the  population  standard 
deviation  for  the  In-Service  sample. 
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In  subsequent  analyses,  the  incremental  validity  coefficients  corrected  for  direct  range 
restriction  on  AFQT  and  shrinkage  constitute  the  more  conservative  “corrected”  coefficients, 
while  those  not  corrected  constitute  the  “uncorrected”  coefficients. 

Incremental  Validity  Results 

The  incremental  validity  results  are  organized  by  the  enlistment  option  and  in-service 
samples.  Each  criterion  is  categorized  as  continuance,  perfonnance,  or  attitudinal  and  results  are 
presented  in  accordance  with  this  categorization.  First,  the  results  of  the  incremental  validity  are 
discussed.  Next  the  results  are  discussed  in  light  of  the  initial  OBEF  validity  findings.  Finally, 
we  discuss  the  specific  scales  that  predict  each  criterion.  Only  significant  results  will  be 
discussed  in  the  text.  Full  results  can  be  found  in  the  respective  tables  and  Appendix  D. 

Enlistment  Option  Results 

Tables  3.1  and  3.2  report  the  incremental  validity  results  of  the  continuous  and 
dichotomous  variables,  respectively,  for  the  enlistment  option  sample.  Appendix  D  presents  the 
bivariate  correlations  for  the  raw  data  and  the  bivariate  correlations  produced  from  the  FIML 
analyses.  AFQT  was  not  a  significant  predictor  of  any  of  the  criterion  measures. 

Continuance  Criteria.  The  Core  RBI  scales  added  incremental  prediction  to  Army 
Commitment  (A R  =  .22).  The  Core  Work  Values  scales  added  incrementally  to  the  prediction  of 
Anny  Commitment  (A R  =  .19)  and  Active  Duty  Separations  (A R  =  .41).  The  SJT  added 
incrementally  to  the  prediction  of  Army  Commitment  (A R  =  .11)  and  Career  Continuance 
Intentions  (A R  =  .23).  In  comparing  these  results  to  the  results  in  the  first  phase  (Russell  & 
Tremble,  2011),  the  Core  RBI  and  Work  Values  scales  were  more  predictive  of  Commitment  and 
Career  Intentions  at  the  End  of  OCS  than  In-Unit.  However,  the  SJT  was  a  better  predictor  of  In- 
Unit  Commitment  and  Career  Intentions  than  at  the  End  of  OCS.  Overall,  the  predictor  sets 
accounted  for  an  additional  1%  to  5%  of  the  variance  in  the  continuance  criteria  beyond  AFQT. 

Looking  closer  at  the  bivariate  correlations10  (see  Appendix  D),  the  Core  RBI  scales  that 
were  significant  predictors  of  continuance  criteria  were  Anny  Affective  Commitment  (r  =  .29), 
Stress  Tolerance  (r  =  .26),  Self-Efficacy  (r  =  .25),  Fitness  Motivation  (r  =  .25),  Peer  Leadership 
(r  =  .19)  and  Achievement  (r  =  .17).  The  Core  Work  values  scales  that  served  as  the  best 
predictors  of  continuance  were  Rugged  Leadership  (r  =  .13  to  .20)  and  Flexibility/Choice  (r  =  - 
.18  to  .22). 


10  Bivariate  correlations  listed  in  text  are  from  the  FIML  estimation. 
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Table  3.1.  Incremental  Validity  Results  of  the  Continuous  Criteria  for  Enlistment  Option  Sample 


Uncorrected  Corrected 


Predictor  Measure/Outcome  Variable 

n 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A  Ra 

A  -2  Log 
Likelihood11 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A R 

Rational  Biodata  Inventory 

Continuance:  Army  Commitment 

368 

.28 

.50 

.22 

29.10 

.35 

.45 

.10 

Performance:  Branch-Specific  Tech  Knowledge 

368 

.03 

.32 

.29 

12.01 

.07 

.21 

.14 

Performance:  Army- Wide  Tech  Knowledge 

368 

.08 

.36 

.28 

17.26 

-.04 

.30 

.33 

Performance:  Physical  Fitness 

368 

.10 

.58 

.48 

47.14 

.09 

.39 

.30 

Performance:  Std.  Total  Awards 

368 

.00 

.25 

.25 

19.83 

-.38 

.21 

.59 

Performance:  Std.  Weighted  Awards 

368 

.03 

.25 

.22 

18.90 

-.02 

.21 

.23 

Attitudinal:  Branch  Satisfaction 

368 

.00 

.30 

.30 

10.37 

-.04 

.24 

.28 

Attitudinal:  Morale 

368 

.19 

.38 

.19 

12.82 

.32 

.38 

.06 

Work  Values 

Continuance:  Army  Commitment 

368 

.28 

.47 

.19 

15.63 

.35 

.48 

.13 

Performance:  Branch-Specific  Tech  Knowledge 

368 

.03 

.44 

.41 

14.04 

.07 

.31 

.25 

Performance:  Army- Wide  Tech  Knowledge 

368 

.08 

.35 

.27 

11.55 

-.04 

.24 

.28 

Performance:  Physical  Fitness 

368 

.10 

.39 

.29 

10.66 

.09 

.20 

.10 

Performance:  Std.  Total  Awards 

368 

.00 

.26 

.26 

10.15 

-.38 

.20 

.58 

Performance:  Std.  Weighted  Awards 

368 

.03 

.24 

.21 

8.93 

-.02 

.20 

.23 

Attitudinal:  Branch  Satisfaction 

368 

.00 

.38 

.38 

10.37 

-.04 

.40 

.43 

Attitudinal:  Morale 

368 

.19 

.39 

.20 

10.51 

.32 

.42 

.11 

Situational  Judgment  Test 

Continuance:  Army  Commitment 

368 

.28 

.39 

.11 

5.11 

.36 

.50 

.15 

Performance:  Branch-Specific  Tech  Knowledge 

368 

.03 

.28 

.25 

4.99 

.07 

.27 

.21 

Performance:  Army- Wide  Tech  Knowledge 

368 

.08 

.08 

.00 

.01 

-.04 

-.08 

-.05 

Performance:  Physical  Fitness 

368 

.10 

.14 

.04 

.42 

.09 

.09 

-.01 

Performance:  Std.  Total  Awards 

368 

.00 

.10 

.10 

1.71 

-.38 

.00 

.38 

Performance:  Std.  Weighted  Awards 

368 

.03 

.15 

.12 

3.63 

-.02 

.13 

.15 

Attitudinal:  Branch  Satisfaction 

368 

.00 

.17 

.17 

1.92 

-.04 

.16 

.20 

Attitudinal:  Morale 

368 

.19 

.39 

.20 

2.24 

.32 

.37 

.05 

Note.  Bolded  results  indicate  significance  at  p<  .05.  J  Bolded  values  under  A R  are  based  on  F-test.  bThe  difference  between  the  -2  Log-likelihood  values  for  the  AFQT  only  model 
and  the  AFQT+  predictor  model;  significant  value  indicates  that  the  AFQT  +  predictor  model  fits  significantly  better  than  the  model  with  AFQT  only.  Uncorrected  Results  are 
based  on  FIML  OLS  regression.  Corrections  were  made  to  the  FIML  correlation  estimates  for  the  range  restriction  on  AFQT  and  theRs  were  corrected  directly  for  shrinkage. 
“Continuance,”  “Performance,”  and  “Attitudinal”  =  types  of  criteria. 


Performance  Criteria.  The  Core  RBI  scales  added  incremental  validity  to  the  prediction 
of  Physical  Fitness  (A R  =  .48)  and  whether  an  Officer  Candidate  graduated  from  initial  training 
(A R  =  .33).  While  the  F-test  was  not  significant  for  Army-Wide  Technical  knowledge  (A R  = 

.28),  Total  Awards  (A R  =  .25),  or  Weighted  Awards  (A R  =  .22)  the  -2  Log-Likelihood  test  was 
significant,  indicating  that  the  addition  of  the  RBI  scale  significantly  improved  the  predictive 
efficacy  of  the  regression  models.  The  Core  Work  Values  scales  added  incremental  validity  to 
the  prediction  of  Anny-Wide  Technical  Knowledge  (A R  =  .27),  Branch-Specific  Technical 
Knowledge  (A R  =  .41),  and  Physical  Fitness  (A R  =  .29).  The  SJT  added  incremental  validity  to 
the  prediction  of  Branch-Specific  Technical  Knowledge  (A R  =  .25).  Leadership,  Personal 
Discipline  and  whether  an  Officer  recycled  during  initial  training  was  not  predicted  by  any  of  the 
experimental  measures.  Overall,  the  Core  RBI  scales  accounted  for  an  additional  8%  to  23%  of 
the  variance  in  the  performance  criteria  over  AFQT,  the  Core  Work  Values  scales  accounted  for 
an  additional  2%  to  17%,  and  the  SJT  accounted  for  an  additional  0%  to  6%. 


Table  3.2.  Incremental  Validity  Results  of  the  Dichotomous  Criteria  for  Enlistment  Option 
Sample 


Pseudo-/?  Estimates 

Predictor  Measure/Outcome  Variable 

n 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A  R 

A  -2  Log 
Likelihood3 

Rational  Biodata  Inventory 

Continuance:  Career  Continuance  Intentions 

368 

.20 

.44 

.24 

12.42 

Continuance:  Active  Duty  Separation 

368 

.04 

.31 

.27 

16.69 

Performance:  Leadership 

368 

.38 

.47 

.08 

11.33 

Performance:  IOT  Recycles 

368 

.17 

.44 

.28 

11.64 

Performance:  IOT  Graduation 

368 

.20 

.53 

.33 

17.07 

Performance:  Personal  Discipline 

368 

.31 

.63 

.32 

12.07 

Work  Values 

Continuance:  Career  Continuance  Intentions 

368 

.20 

.43 

.23 

4.42 

Continuance:  Active  Duty  Separation 

368 

.04 

.46 

.41 

22.51 

Performance:  Leadership 

368 

.38 

.51 

.13 

3.82 

Performance:  IOT  Recycles 

368 

.17 

.30 

.14 

-2.08 

Performance:  IOT  Graduation 

368 

.20 

.46 

.26 

1.04 

Performance:  Personal  Discipline 

368 

.31 

.64 

.33 

3.65 

Situational  Judgment  Test 

Continuance:  Career  Continuance  Intentions 

368 

.20 

.43 

.23 

7.20 

Continuance:  Active  Duty  Separation 

368 

.04 

.05 

.01 

.17 

Performance:  Leadership 

368 

.38 

.38 

.00 

.30 

Performance:  IOT  Recycles 

368 

.17 

.24 

.07 

.91 

Performance:  IOT  Graduation 

368 

.20 

.24 

.04 

.68 

Performance:  Personal  Discipline 

368 

.31 

.31 

.00 

.18 

Note.  Bolded  results  indicate  significance  at  p<  .05.  aThe  difference  between  the  -2  Log-likelihood  values  for  the  AFQT  only 
model  and  the  AFQT+  predictor  model;  significant  value  indicates  that  the  AFQT  +  predictor  model  fits  significantly  better  than 
the  model  with  AFQT  only.  Results  are  based  on  FIML  OLS  regression.  “Continuance”  and  “Performance”  =  types  of  criteria. 


Examining  the  Core  Work  Values  scales  across  all  of  the  perfonnance  criteria  (see  Appendix 
D),  Rugged  Leadership  (r  =19  -  .31;  mean  r  =  .23)  showed  the  strongest  relationships  with  the 
perfonnance  criteria.  For  the  Core  RBI  scales,  Peer  Leadership  (r  =.  19  -.25,  mean  r  =.22),  Fitness 
Motivation (r  =  .15  -  .54;  meanr  =  .35),  Self-Efficacy  (r  =  .15  -  .19,  meanr=  .17),  Achievement  (r  = 
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.13  -  .19,  mean  r  =  .16)  and  Continuance  Commitment  (r  =  -.17  to  -.10;  mean  r  =  -.14)  showed  the 
strongest  relationships  with  Anny-Wide  Technical  Knowledge  and  Physical  Fitness. 

The  In-Unit  perfonnance  criteria  were  often  measured  differently  from  the  criteria  used  at 
the  End  of  OCS;  however,  in  many  cases  the  same  criterion  construct  was  measured,  suggesting 
meaningful  comparisons  could  be  made.  We  examined  the  validity  coefficients  reported  in  the 
validation  chapter  of  the  initial  report  (Allen,  201 1)  and  compared  them  to  those  derived  in  the 
current  analyses.  At  the  End  of  OCS,  the  Core  RBI  scales  add  incrementally  to  the  prediction  of 
OCS  Fitness  perfonnance,  APFT,  and  overall  OCS  performance  (mean  A R  =  .46)  and 
demonstrated  similar  incremental  efficacy  for  physical  fitness  and  technical  in-unit  perfonnance 
(mean  A R  =  .35).  The  Core  Work  Values  did  not  add  incrementally  to  any  perfonnance  criteria  at 
the  End  of  OCS.  However,  the  Core  Work  Values  were  strong  predictors  of  In-Unit  technical 
knowledge  and  physical  fitness,  suggesting  that  Core  Work  Values  may  be  a  better  long  tenn 
predictor  of  performance.  Finally,  the  SJT  showed  some  value  for  predicting  technical 
perfonnance  criteria  both  at  the  End  of  OCS  (OCS  Academic  Performance)  and  In-Unit  (Branch 
Specific  Technical  Knowledge).  While  none  of  the  scales  added  incremental  validity  to  the 
prediction  of  In-Unit  Leadership,  the  Core  RBI  scales  added  incremental  validity  to  the 
prediction  of  OCS  Leadership  Performance.  These  analyses  suggest  that  the  Core  RBI  scales 
could  be  used  to  predict  both  short-term  and  longer-term  performance  of  officers.  However,  the 
Core  Work  Value  scales  are  better  suited  to  predicted  longer-tenn  perfonnance. 

Attitudinal  Criteria.  The  Core  Work  Values  scales  added  incremental  validity  to  the 
prediction  of  Branch  Satisfaction  (A R  =  .38)  and  Morale  (A R  =  .20),  accounting  for  an  additional 
14%  and  4%  of  the  variance,  respectively.  The  Core  RBI  scales  and  the  SJT  did  not  predict  the 
attitudinal  criteria.  These  results  are  not  consistent  with  the  results  reported  in  Russell  and 
Tremble  (2011),  where  the  RBI  added  incrementally  to  the  prediction  of  Morale  and  the  Core 
Work  Values  scales  did  not  add  incrementally  to  the  prediction  of  Branch  Satisfaction.  However, 
the  Core  Work  Values  added  incrementally  to  the  prediction  of  Morale  both  at  the  End  of  OCS 
and  In-Unit. 

The  Core  Work  Values  Rugged  Leadership  (r  =  .23),  Skill  Development  (r  =  .21),  and 
Teamwork  (r  =  .24)  were  significantly  related  to  Morale,  while  Core  Work  Values 
Altruism/Benevolence  (r  =  .16)  and  Job  Security  (r  =  -.21)  were  significantly  related  to  Branch 
satisfaction. 

Summary  of  incremental  validity  results  for  the  enlistment  option  sample: 

•  The  Core  RBI  scales  had  stronger  relationships  with  In-Unit  continuance  criteria  than 
the  Core  Work  Values  scales  or  SJT. 

•  The  Core  Work  Values  scales  were  the  only  predictors  to  add  incrementally  to  the 
prediction  of  actual  Separations. 

•  Both  the  Core  RBI  scales  and  Core  Work  Values  scales  showed  strong  relationships 
with  the  performance  criteria. 

•  The  Core  Work  Values  scales  showed  stronger  relationships  with  the  attitudinal 
criteria  than  the  RBI  or  SJT. 
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•  The  Core  Work  Values  scales  generally  added  less  incremental  validity  to  the 
prediction  of  the  criteria  at  the  End  of  OCS,  but  showed  greater  predictive  utility 
among  the  In-Unit  criteria. 

•  The  Core  RBI  scales  showed  more  consistency  in  predicting  the  performance  criteria 
at  the  End  of  OCS  and  In-Unit  than  the  Core  Work  Values  scales  or  SJT. 

•  Among  the  Core  Work  Values  scales,  the  Rugged  Leadership  (mean  r  =  .20)  and 
Teamwork  (mean  r  =  .15)  scales  showed  the  strongest  correlations  across  all  criteria. 

•  Several  Core  RBI  scales  exhibited  statistically  significant  correlations  across  the 
criteria  of  interest,  with  the  highest  magnitude  scales  being  Fitness  Motivation  (mean 
r  -  .22),  Stress  Tolerance  (mean  r  =  .18),  Self-Efficacy,  (mean  r=  .16),  and  Peer 
Leadership  (mean  r  =  .14). 

In-Service  Results 

Tables  3.3  and  3.4  report  the  incremental  validity  results  of  the  continuous  and 
dichotomous  variables,  respectively,  for  the  in-service  sample.  Appendix  D  presents  the  bivariate 
correlations  for  the  raw  data  and  the  bivariate  correlations  produced  from  the  FIML  analyses. 
AFQT  was  not  a  significant  predictor  of  any  of  the  criterion  measures.  AFQT  accounted  for 
between  one  and  four  percent  of  the  variance  in  the  In-Unit  criteria. 

Continuance  Criteria.  The  Core  Work  Values  scales  added  incremental  validity  to  the 
prediction  of  Army  Commitment  (A R  =  .25)  and  Career  Intentions  (Mi  =  .50),  while  the  Core 
RBI  scales  added  incrementally  to  the  prediction  of  Army  Commitment  (A R  =  .32).  The  Core 
Work  Values  and  Core  RBI  scales  accounted  for  an  additional  6%  to  25%  of  the  variance  in 
continuance  criteria.  The  SJT  did  not  add  incremental  validity  to  either  of  the  continuance 
criteria  and  Separations  were  not  predicted  by  any  of  the  predictor  measures.  In  comparing  these 
results  to  the  results  at  the  End  of  OCS  (Russell  &  Tremble,  2011),  the  RBI,  Work  Values,  and 
SJT  were  generally  more  predictive  of  commitment  at  the  End  of  OCS  than  In-Unit.  However, 
the  Core  Work  Values  scales  were  more  predictive  of  In-Unit  Career  Intentions  than  Officer 
Career  Intentions  at  the  End  of  OCS. 

Peer  Leadership  (r  =  .21),  Achievement  (r  =  .26)  and  Army  Affective  Commitment  (r  = 
.45)  stood  out  as  the  driving  predictors  of  Anny  Commitment  among  the  Core  RBI  scales.  Of  the 
Work  Value  scales,  Rugged  Leadership  (r  =.26),  Structure/Recognition  (r  =  .20),  Skill 
Development  (r  =  .3 1),  Altruism/Benevolence  (r  =  .26)  and  Teamwork  (r  =  .36)  emerged  as  the 
driving  predictors  of  the  continuance  criteria. 
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Table  3.3.  Incremental  Validity  Results  of  the  Continuous  Criteria  for  the  In-Service  Sample 


Uncorrected  Corrected 


Predictor  Measure/Outcome  Variable 

n 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A  It 

A  -2  Log 
Likelihood*5 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A  R 

Rational  Biodata  Inventory > 

Continuance:  Army  Commitment 

390 

.19 

.51 

.32 

43.64 

.23 

.45 

.22 

Performance:  Branch-Specific  Tech  Knowledge 

390 

.13 

.34 

.21 

16.30 

.25 

.39 

.15 

Performance:  Army- Wide  Tech  Knowledge 

390 

.07 

.33 

.26 

15.74 

.12 

.33 

.21 

Performance:  Physical  Fitness 

390 

.07 

.42 

.35 

29.72 

.07 

.40 

.33 

Attitudinal:  Branch  Satisfaction 

390 

.06 

.37 

.30 

25.11 

.09 

.30 

.21 

Attitudinal:  Morale 

390 

.15 

.40 

.25 

25.98 

.18 

.35 

.17 

Work  Values 

Continuance:  Army  Commitment 

390 

.19 

.44 

.25 

20.59 

.23 

.36 

.12 

Performance:  Branch-Specific  Tech  Knowledge 

390 

.13 

.17 

.04 

1.01 

.25 

.30 

.06 

Performance:  Army-Wide  Tech  Knowledge 

390 

.07 

.42 

.35 

10.15 

.12 

.28 

.16 

Performance:  Physical  Fitness 

390 

.07 

.39 

.32 

11.60 

.07 

.34 

.27 

Attitudinal:  Branch  Satisfaction 

390 

.07 

.26 

.19 

5.97 

.09 

.29 

.20 

Attitudinal:  Morale 

390 

.15 

.47 

.32 

19.74 

.18 

.47 

.29 

Situational  Judgment  Test 

Continuance:  Army  Commitment 

390 

.19 

.30 

.11 

1.69 

.23 

.25 

.02 

Performance:  Branch-Specific  Tech  Knowledge 

390 

.13 

.14 

.01 

.06 

.25 

.25 

.01 

Performance:  Army- Wide  Tech  Knowledge 

390 

.07 

.08 

.01 

.04 

.12 

.13 

.01 

Performance:  Physical  Fitness 

390 

.06 

.16 

.09 

1.13 

.07 

.21 

.15 

Attitudinal:  Branch  Satisfaction 

390 

.06 

.06 

.00 

.00 

.09 

.07 

-.02 

Attitudinal:  Morale 

390 

.15 

.20 

.05 

.87 

.18 

.30 

.13 

Note.  Bolded  results  indicate  significance  at  />  <  .05.  a  Bolded  values  under  A R  are  based  on  F-test.  b  The  difference  between  the  -2  Log-likelihood  values  for  the  AFQT  only  model 
and  the  AFQT+  predictor  model;  significant  value  indicates  that  the  AFQT  +  predictor  model  fits  significantly  better  than  the  model  with  AFQT  only.  Uncorrected  results  are 
based  on  FIML  OLS  regression.  Corrections  were  made  to  the  FIML  correlation  estimates  for  the  range  restriction  on  AFQT  and  the  Rs  were  corrected  directly  for  shrinkage. 
“Continuance,”  “Performance,”  and  “Attitudinal”  =  types  of  criteria. 


Performance  Criteria.  The  Core  RBI  scales  added  incremental  validity  to  the  prediction 
of  Anny-Wide  Technical  Knowledge  (A R  =  .26),  Physical  Fitness  (A R  =  .35),  Personal 
Discipline  (A R  =  .53),  and  whether  an  Officer  recycled  during  initial  training  (AP  =  .52).  The 
Core  RBI  scales  accounted  for  an  additional  7%  to  27%  of  the  variance  in  the  three  performance 
criteria.  These  results  are  consistent  with  the  Russell  and  Tremble  (2011)  results,  where  the  RBI 
added  incrementally  to  the  prediction  of  Academic  Perfonnance,  Fitness  Performance,  and 
overall  OCS  Perfonnance  (mean  r  =  27).  The  Core  Work  Values  scales  added  incremental 
validity  to  the  prediction  of  Army-Wide  Technical  Knowledge  (AR  =  .35)  and  Physical  Fitness 
(AR  =  .32),  accounting  for  an  addition  10%  and  12%  of  the  variance.  These  results  are 
inconsistent  with  the  results  from  the  first  phase  of  this  research.  The  Core  Work  Values  scales 
added  more  incrementally  to  the  prediction  of  In-Unit  perfonnance  criteria  compared  to  similar 
criteria  at  the  End  of  OCS.  The  SJT  did  not  add  incremental  validity  to  either  of  the  continuance 
criteria.  Branch-Specific  Technical  Knowledge,  Leadership,  and  whether  an  Officer  candidate 
graduated  from  initial  training  were  not  predicted  by  any  of  the  experimental  measures. 

Table  3.4.  Incremental  Validity  Results  of  the  Dichotomous  Criteria  for  the  In-Service  Sample 

Pseudo -A1  Estimates 


Predictor  Measure/Outcome  Variable 

n 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A  R 

A  -2  Log 
Likelihood3 

Rational  Biodata  Inventory 

Continuance:  Career  Continuance  Intentions 

390 

.14 

.52 

.38 

11.97 

Continuance:  Active  Duty  Separation 

390 

.18 

.44 

.26 

14.40 

Performance:  Leadership 

390 

.10 

.24 

.14 

4.52 

Performance:  IOT  Recycles 

390 

.08 

.60 

.52 

18.97 

Performance:  IOT  Graduation 

390 

.11 

.64 

.53 

12.12 

Performance:  Personal  Discipline 

390 

.40 

.93 

.53 

21.47 

Work  Values 

Continuance:  Career  Continuance  Intentions 

390 

.15 

.65 

.50 

14.55 

Continuance:  Active  Duty  Separation 

390 

.20 

.41 

.21 

5.84 

Performance:  Leadership 

390 

.09 

.52 

.42 

10.68 

Performance:  IOT  Recycles 

390 

.08 

.72 

.64 

12.88 

Performance:  IOT  Graduation 

390 

.11 

.39 

.28 

-.19 

Performance:  Personal  Discipline 

390 

.40 

.65 

.25 

6.12 

Situational  Judgment  Test 

Continuance:  Career  Continuance  Intentions 

390 

.15 

.33 

.18 

-.43 

Continuance:  Active  Duty  Separation 

390 

.20 

.20 

.00 

-.68 

Performance:  Leadership 

390 

.09 

.15 

.05 

-.98 

Performance:  IOT  Recycles 

390 

.08 

.33 

.25 

2.18 

Performance:  IOT  Graduation 

390 

.12 

.37 

.25 

.23 

Performance:  Personal  Discipline 

390 

.40 

.80 

.40 

1.73 

Note.  Bolded  results  indicate  significance  at p<  .05.  aThe  difference  between  the  -2  Log-likelihood  values  for  the  AFQT  only 
model  and  the  AFQT  +  predictor  model;  a  bolded  value  indicates  that  the  AFQT  +  predictor  model  fits  significantly  better  than 
the  model  with  AFQT  only.  The  pseudo-i?  estimates  are  based  on  FIML  logistic  regression.  “Continuance”  and  “Perfonnance”  = 
types  of  criteria. 


The  Core  Work  Values  scales  Rugged  Leadership  (r  =  .27,  .29),  Altruism/Benevolence  (r 
=  .25,  .24),  Structure/Recognition  (r  =  .14,  .19)  and  Teamwork  (r  =  .21,  .13)  showed  the 
strongest  relationships  with  Physical  Fitness  and  Army-Wide  Technical  Knowledge, 
respectively.  Among  the  Core  RBI  scales,  Fitness  Motivation  (r  =  .19,  .38)  and  Self-Efficacy  (r 
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=  .24,  .11)  showed  the  strongest  relationships  with  Physical  Fitness  and  Army-Wide  Technical 
Knowledge,  respectively. 

Attitudinal  Criteria.  The  Core  RBI  scales  added  incremental  validity  to  the  prediction  of 
Branch  Satisfaction  (A R  =  .30),  and  Morale  (A R  =  .25),  accounting  for  an  additional  9%  and  6% 
of  the  variance  in  the  respective  attitudinal  criteria.  The  Core  Work  Values  scales  only  added 
incremental  validity  to  the  prediction  of  Morale  (A  R  =  .32).  The  SJT  did  not  add  incremental 
validity  to  any  of  the  attitudinal  criteria.  Comparing  these  results  to  those  found  in  Russell  and 
Tremble  (2011),  the  Core  RBI  and  Work  Values  scales  predicted  Morale  similarly  at  the  End  of 
OCS,  but  the  Core  RBI  was  a  better  predictor  of  In-Unit  Branch  Satisfaction. 

Examining  the  bivariate  correlations  of  the  Core  RBI  scales  show  that  Achievement  (r  = 
.17,  .25),  Stress  Tolerance  (r  =  .22,  .16),  Self-Efficacy  (r  =  .20,  .32),  and  Army  Affective 
Commitment  (r  =  .25,  .30)  were  the  strongest  predictors  of  Branch  Satisfaction  and  Morale. 
Several  Work  Values  scales  were  highly  correlated  with  Morale,  including  Rugged  Leadership  (r 
=  .33),  Altruism/Benevolence  (r  =  .20),  Skill  Development  (r  =  .32),  Job  Security  (r  =  .21),  and 
Teamwork  (r  =  .33). 

Summary  of  incremental  validity  results  for  the  in-service  sample: 

•  Among  all  of  the  Core  RBI  scales,  Peer  Leadership  (mean  r  =  .16),  Fitness 
Motivation  (mean  r  =  .  14),  Self-Efficacy  (mean  r  =  .20)  and  Anny  Affective 
Commitment  (mean  r  =  .23)  showed  the  strongest  average  relationships  with  the 
criteria. 

•  With  the  exception  of  the  Flexibility/Choice  scale,  the  Core  Work  Values  scales 
generally  showed  significant  relationships  with  the  perfonnance  (mean  r  =  .14), 
continuance  (mean  r  =  .26),  and  attitudinal  criteria  (mean  r  =  .21). 

•  Overall,  both  the  Core  RBI  scales  and  the  Core  Work  Values  scales  predicted  the 
continuance  and  attitudinal  criteria  well  for  the  in-service  sample. 

•  For  the  most  part,  the  Core  RBI  and  Work  Value  scales  afforded  similar  or  better 
predictive  utility  for  the  In-Unit  criteria  compared  to  the  criteria  at  the  End  of  OCS. 

OBEF  Predictor  Composites 

During  the  validation  of  the  predictor  measures  at  the  End  of  OCS  (Allen,  2011), 
composites  were  identified  in  an  effort  to  combine  predictor  scales  that  would  optimize  the 
prediction  of  end-of-OCS  criteria.  The  in-service  OBEF  composite  was  comprised  of  five 
positively  valenced  scales  and  two  negatively  valenced  scales.  The  in-service  composite  is 
represented  with  the  following  formula: 

In-Service  OBEF  Composite  =  RBI  Achievement  +  RBI  Fitness  Motivation  +  RBI  Stress 

Tolerance  +  RBI  Self-Efficacy  +  Work  Values  Skill  Development  -  Work  Values 

Flexibility/Choice  -  Work  Values  Structure/Recognition 
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The  enlistment  option  OBEF  composite  included  four  positively  valenced  scales  and  three 
negatively  valenced  scales.  The  composite  is  represented  with  the  following  formula: 

Enlistment  Option  OBEF  Composite  =  RBI  Achievement  +  RBI  Fitness  Motivation  + 

RBI  Stress  Tolerance  +  Work  Values  Altruism/Benevolence  -  RBI  Hostility  to  Authority 

-  Work  Values  Flexibility/Choice  -  Work  Values  Structure/Recognition 

The  composites  identified  using  the  End-of-OCS  criteria  were  examined  to  assess  how  well  the 
composites  predicted  the  In-Unit  criteria.  Table  3.5  presents  the  results  for  both  samples. 

Continuance  Criteria.  Both  the  in-service  composite  and  the  enlistment  option  composite 
added  incremental  validity  to  the  prediction  of  Army  Commitment  (A R  =  .21;  A R  =  .23), 
respectively.  The  enlistment  option  composite  added  incrementally  to  the  prediction  of 
Separations  (A R  =  .42).  Career  Intentions  was  not  predicted  by  the  composite  for  either  sample. 

Performance  Criteria.  The  in-service  composite  added  incrementally  to  the  prediction  of 
Army-Wide  Technical  Knowledge  (A R  =  .24)  and  Physical  Fitness  (A R  =  .37).  For  the 
enlistment  option  sample,  the  composite  added  incrementally  to  the  prediction  of  Physical 
Fitness  (A R  =  .47)  and  whether  an  Officer  candidate  graduated  from  initial  training  (A R  =  .34). 
Branch-Specific  Technical  Knowledge,  Leadership,  Awards,  and  whether  an  Officer  recycled 
during  initial  training  were  not  predicted  by  the  composite  for  either  sample. 

Attitudinal  Criterion.  For  the  in-service  sample,  the  composite  added  incremental  validity 
to  the  prediction  of  Branch  Satisfaction  (A R  =  .33)  and  Morale  (AR  =  .30).  The  enlistment  option 
did  not  add  incremental  validity  to  the  prediction  of  any  of  the  attitudinal  criterion. 

Conclusion 

The  results  of  the  incremental  validity  analyses  showed  significant  relationships  between 
the  Core  predictor  scales  and  the  perfonnance  and  continuance  outcomes.  Of  all  the  criterion 
variables,  Army  Commitment  was  best  predicted,  while  Leadership  and  Branch-specific  job- 
knowledge  were  poorly  predicted.  For  the  in-service  sample,  the  Core  RBI  scales,  the  Core  Work 
Values  scales,  and  the  OBEF  composites  were  all  significant  predictors  of  continuance, 
performance,  and  attitudinal  outcomes.  For  the  enlistment  option  sample,  the  Core  Work  Values 
scales  were  the  strongest  predictors  of  outcomes,  however  there  were  a  number  of  outcomes 
predicted  by  the  Core  RBI  scales  and  the  OBEF  Composites.  The  SJT  was  not  a  particularly 
good  predictor;  there  were  no  significant  relationships  for  the  in-service  sample.  However,  for 
the  enlistment  option  sample  the  SJT  predicted  Branch-Specific  Knowledge  and  Career 
Intentions,  neither  of  which  were  predicted  by  the  RBI  or  Work  Values  scales.  The  SJT  may 
offer  some  utility  in  addition  to  the  RBI  and  Work  Values  scales,  but  only  for  the  enlistment 
option  sample. 
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Table  3.5.  Incremental  Validity  of  the  OBEF  Predictor  composite  for  In-Service  and  Enlistment  Option  Incremental  Validity  of  the 
OBEF  Predictor  Composites  for  the  In-Service  and  Enlistment  Option  Samples 


OBEF  Predictor  Composite 

In-Service 

Enlistment  Option 

n 

AFQT 
Only  R 

AFQT  + 

Exp  R 

A  R 

A  -2  Log 
Likelihood3 

n 

AFQT 
Only  R 

AFQT  + 
Exp  R 

A R 

A  -2  Log 
Likelihood3 

Continuous  Criterion 

Army  Commitment1 

390 

.19 

.41 

.21 

20.59 

368 

.28 

.51 

.23 

25.64 

Branch-Specific  Technical  Knowledge2 

390 

.13 

.26 

.14 

7.99 

368 

.03 

.28 

.25 

7.62 

Army-Wide  Technical  Knowledge2 

390 

.07 

.31 

.24 

12.21 

368 

.08 

.33 

.25 

10.38 

Physical  Fitness2 

390 

.07 

.44 

.37 

29.47 

368 

.10 

.57 

.47 

44.66 

Std.  Total  Awards2 

— 

— 

— 

— 

— 

368 

.00 

.23 

.23 

12.82 

Std.  Weighted  Awards2 

— 

— 

— 

— 

— 

368 

.03 

.22 

.19 

11.82 

Branch  Satisfaction2 

390 

.07 

.40 

.33 

22.20 

368 

.00 

.31 

.31 

8.75 

Morale3 

390 

.15 

.46 

.30 

26.56 

368 

.19 

.31 

.12 

7.78 

Dichotomous  Criterion 

Career  Continuance  Intentions' 

390 

.15 

.47 

.32 

10.25 

368 

.20 

.47 

.27 

12.52 

Active  Duty  Separation1 

390 

.20 

.31 

.11 

-.80 

368 

.04 

.46 

.42 

27.73 

Leadership2 

390 

.09 

.21 

.12 

1.06 

368 

.38 

.50 

.12 

12.10 

IOT  Recycles2 

390 

.08 

.72 

.64 

13.46 

368 

.17 

.44 

.27 

10.12 

IOT  Graduation2 

390 

.12 

.76 

.64 

12.31 

368 

.20 

.54 

.34 

14.37 

Personal  Discipline2 

390 

.41 

.61 

.20 

4.32 

368 

.31 

.62 

.31 

9.34 

Note.  Bolded  results  indicate  significance  at p<  .05.  aThe  difference  between  the  -2  Log-likelihood  values  for  the  AFQT  only  model  and  the  AFQT  +  predictor  model;  a  bolded 
value  indicates  that  the  AFQT  +  predictor  model  fits  significantly  better  than  the  model  with  AFQT  only.  Results  are  based  on  FIML  OLS  regression.  'Continuance  Criteria, 
'Performance  Criteria,  ’Attitudinal  Criteria. 
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For  the  purposes  of  the  longitudinal  validation  research,  we  collected  data  from  and 
tracked  officers  across  four  time  points.  As  discussed  in  Chapter  2,  those  time  points  included 
the  Beginning  of  OCS,  the  End  of  OCS,  BOLC  III/B,  and  In-Unit  (near  the  end  of  the  active  duty 
service  obligation  [ADSO]).  Several  attitudinal  measures  were  administered  three  or  more  of 
these  time  points,  allowing  a  unique  opportunity  to  examine  how  these  variables  change  over 
time,  the  antecedents  of  these  changes,  and  their  consequences. 

Variables  such  as  Career  Intentions,  Commitment,  and  Morale  have  previously  been  used 
as  proxy  criteria  to  validate  new  selection  measures,  particularly  non-cognitive  selection 
measures.  Career  Intentions  and  Affective  Commitment,  in  particular,  have  been  a  focus  of  the 
Army  as  the  Anny  looks  to  identify  highly-qualified  officer  candidates  who  will  continue  service 
beyond  their  initial  obligations.  Understanding  how  attitudes  change  over  time,  whether  there  are 
factors  that  influence  how  attitudes  change  over  time  and  the  consequences  of  changes  in 
attitudes  is  an  important  research  topic.  The  current  analysis  seeks  to  investigate  these  issues  by 
answering  the  following  questions: 

•  How  do  Officer  Commitment,  Career  Intentions,  Branch  Satisfaction,  and  Morale 
change  over  the  first  three  years  of  post-commissioning  service? 

•  Are  these  propensities  for  change  different  for  enlistment  option  and  in-service  samples? 

•  Can  individual  differences  (measured  through  the  OBEF)  be  used  to  identify  officers  who 
have  a  higher  propensity  to  be  more  satisfied  and  committed  to  the  Army  over  time? 

•  What  are  the  consequences  of  attitudes  decreasing  or  increasing  over  time?  For  example, 
if  Morale  increases,  do  officers  show  higher  commitment,  stronger  Career  Intentions,  and 
less  instances  of  separation? 

The  remainder  of  this  chapter  outlines  the  specific  analyses  used  to  examine  longitudinal 
change  over  time  and  the  associated  results.  Finally,  we  discuss  the  implications  and  how  the 
results  can  contribute  to  our  understanding  of  officer  candidate  selection. 

Approach 

We  examined  change  in  five  attitudinal  variables:  Affective  Commitment,  Continuance 
Commitment,  Career  Intentions,  Morale,  and  Satisfaction  with  Branch  Assignment.  Table  4. 1 
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provides  a  brief  description  of  each  variable  and  a  list  of  time  points  when  each  variable  was 
measured.  Note  that  these  variables  are  different  from  those  described  in  Chapter  2.  The 
variables  described  in  Chapter  2  are  constructs  derived  from  one  or  more  measures,  whereas  the 
variables  in  this  chapter  are  specific  scales  that  were  measured  at  the  given  time  points.  The  Core 
RBI  scales,  Core  Work  Values  scales,  SJT,  and  the  selection  composites  developed  in  the  first 
phase  of  this  research  were  examined  as  predictors  of  change.  Separation,  In-Unit  career 
intentions,  and  awards  were  examined  as  outcomes  of  change. 


Table  4.1.  Description  of  Longitudinal  Variables. 


Scale 

Description 

Measurement  Occasions 

Continuance 

Commitment 

Four-item  scale  measuring  the  extent  to  which  the  • 

candidate  was  committed  to  completing  their  • 

Active  Duty  Service  Obligation  (ADSO).  An 
example  item  is  “It  would  be  too  costly  for  me  to  • 

leave  the  Army  in  the  near  future.”  Items  were 
scored  on  a  1  to  5  scale  ranging  from  “Strongly 
Disagree”  to  “Strongly  Agree.”  Administered  with 
Version  1  of  the  OBEF  and  both  end-of-class 
surveys. 

Beginning  ofOCS  (Time  1) 

End  ofOCS  (Time  2) 

BOLC  III/B  (Time  3) 

In  Unit/  Prior  to  end  of  ADSO  (Time 

4) 

Affective 

Commitment 

Four-item  scale  measuring  the  extent  to  which  a  • 

candidate  felt  emotionally  attached  to  the  Army.  • 

An  example  item  is  “1  feel  like  ‘part  of  the  family’  • 
in  the  Army.”  Items  were  scored  on  a  1  to  5  scale  • 
ranging  from  “Strongly  Disagree”  to  “Strongly 

Agree.”  Administered  with  all  versions  of  the 

OBEF  and  both  end-of-class  surveys. 

Beginning  ofOCS  (Time  1) 

End  ofOCS  (Time  2) 

BOLC  III/B  (Time  3) 

In  Unit/  Prior  to  end  of  ADSO  (Time 

4) 

Morale 

A  single-item  measure  of  a  candidate’s  current  • 

level  of  morale  (i.e..  “What  is  vour  current  level  of  • 
morale?”).  The  item  was  scored  on  a  1  to  5  scale  • 

ranging  from  “Very  Low”  to  “Very  High.” 
Administered  with  both  versions  of  the  end-of- 
class  surveys. 

End  ofOCS  (Time  2) 

BOLC  III/B  (Time  3) 

In  Unit/  Prior  to  end  of  ADSO  (Time 

4) 

Career 

Intentions 

A  single-item  measure  of  a  candidate’s  active  duty  • 
career  intentions  (i.e.,  “What  are  your  current  • 

active  duty  career  intentions?”).  The  item  was  • 

scored  on  a  1  to  5  scale  ranging  from  “1  will 
definitely  quit  the  Army  upon  completion  of  my 
obligation”  to  “I  plan  to  stay  in  the  Army  beyond 

20  years  or  until  retirement.”  Administered  with  all 
versions  of  the  OBEF  and  both  versions  of  the  end- 
of-class  surveys. 

End  ofOCS  (Time  2) 

BOLC  III/B  (Time  3) 

In  Unit/  Prior  to  end  of  ADSO  (Time 

4) 

Branch 

Satisfaction 

A  single-item  measure  of  a  candidate’s  satisfaction  • 
with  branch  assignment  (i.e.,  “How  satisfied  are  • 

you  with  your  branch  assignment?”).  The  item  was  • 
scored  on  a  1  to  5  scale  ranging  from  “Very 

Satisfied”  to  “Very  Dissatisfied”. 

End  ofOCS  (Time  2) 

BOLC  III/B  (Time  3) 

In  Unit/  Prior  to  end  of  ADSO  (Time 

4) 
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Latent  Growth  Modeling  (LGM),  an  application  of  confirmatory  factor  analysis,  was 
used  to  model  change  over  time.  Repeated  measures  of  a  variable  over  time  are  used  as  the 
manifest  indicators  of  two  (for  linear  change)  or  three  (for  non-linear  change)  latent  variables:  (a) 
initial  status,  (b)  change,  and  (c)  quadratic  change.  We  used  Mplus  LGM  analyses  with  Full 
Infonnation  Maximum  Likelihood  (FIML)  missing  data  (Muthen  &  Muthen,  2004).  As  discussed 
in  Chapter  3,  FIML  uses  all  available  information  to  estimate  parameters,  even  when  data  are 
missing.  Officers  with  at  least  two  time  points  on  the  longitudinal  measures  were  included  in  the 
analyses.  We  used  three  stages  to  assess  the  change  in  each  variable  and  to  examine  the 
predictors  and  outcomes  of  change: 

•  Stage  1 :  Examine  the  type  of  change  that  characterized  the  data,  linear  or  non-linear. 

•  Stage  2:  Assess  whether  the  change  trajectory  for  the  in-service  and  enlistment  option 
samples  are  equivalent. 

•  Stage  3:  Examine  whether  there  is  variance  associated  with  the  change  trajectories  across 
individuals  and  model  predictors  and  outcomes  of  change. 


Stage  1.  To  examine  linear  change,  two  latent  variables  were  specified  for  each  LGM  - 
initial  status  and  change  -  and  the  observed  variable  for  each  measurement  occasion  was 
included  as  a  manifest  indicator  of  the  two  latent  variables.  Initial  status  represents  the  mean 
starting  value  of  the  variable  of  interest  across  all  individuals.  Change  represents  the  mean 
change  in  the  variable  across  all  individuals.  In  order  to  model  the  latent  growth,  the  factor 
loadings  for  each  of  the  four  measurement  occasions  were  constrained  to  be  one  for  the  initial 
status  latent  construct  (see  Figure  1).  For  the  change  latent  construct,  the  time  one  factor  loading 
was  constrained  to  zero,  the  time  two  factor  loading  was  constrained  to  one,  the  time  three  factor 
loading  was  constrained  to  two,  and  the  time  four  (if  applicable)  factor  was  constrained  to  three. 
To  examine  non-linear  change,  an  additional  factor  was  added  to  the  model,  quadratic  change. 
The  factor  loadings  were  the  same  as  described  above  for  the  initial  status  and  linear  change 
constructs.  For  the  quadratic  latent  construct,  the  factor  loadings  for  time  one,  two,  three,  and 
four  were  constrained  to  0,  1 ,  4,  and  9,  respectively. 

The  model  produces  goodness-of-fit  estimates  that  assess  overall  model  fit  and  mean  and 
variance  estimates  of  the  initial  status  and  change  constructs.  Overall  model  fit  was  assessed 
using  the  %2  statistic,  the  comparative  fit  index  (CFI),  the  Tucker-Lewis  index  (TLI),  the  root 
mean  square  error  of  approximation  (RMSEA)  and  standardized  root  mean  square  residual 
(SRMR).  CFI  and  TLI  values  >  .95  and  SRMR  and  RMSEA  values  <  .08  indicate  acceptable  fit. 
A  significant  mean  change  estimate  suggests  significant  growth  over  time  in  the  attitudinal 
variable.  Significant  variance  estimates  suggest  individual  differences  in  change  over  time. 
Critical  z-scores  were  used  to  test  significance  of  the  mean  and  variance  parameters.  To 
determine  the  best  fitting  growth  pattern,  the  significance  values  of  the  latent  mean  estimates  for 
linear  change  and  for  quadratic  change  were  compared.  Note  that  the  results  for  this  stage  are  not 
reported  for  Morale  due  to  the  sensitivity  concerns. 
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Figure  4.1.  Latent  Growth  Model  of  Change  in  Commitment  with  Core  RBI  as  a  predictor  of  change,  and  Separation  as  an 
outcome  of  change. 


Stage  2.  A  series  of  steps  were  taken  to  examine  the  differences  in  change  between  the 
in-service  and  enlistment  option  samples.  Multiple  groups  LGM  was  conducted  where,  within 
the  same  overall  model,  separate  covariance  matrices  and  model  parameters  are  estimated  for 
each  group.  Using  this  method,  the  change  and  initial  status  can  be  constricted  to  be  equal  to 
compare  whether  the  parameter  estimates  for  each  group  are  similar  or  different.  First,  a  model  is 
estimated  where  the  change  and  initial  status  for  each  group  is  free  to  vary.  To  detennine  if  the 
mean  initial  status  is  significantly  different  between  groups,  a  LGM  is  constructed  where  the 
initial  status  for  each  group  is  constrained  to  be  equal.  The  fit  of  the  constrained  and 
unconstrained  models  are  compared  using  a  chi-square  difference  test.  If  the  models  are 
significantly  different,  where  the  model  with  the  unconstrained  estimate  fits  significantly  better, 
then  we  can  conclude  there  are  differences  between  the  two  groups  initial  status.  If  there  is  no 
difference  in  the  model  fit  between  the  unconstrained  and  the  constrained  models,  the  more 
parsimonious  model  is  considered  to  be  the  better  fitting  and  we  proceed  with  the  model  where 
the  two  groups’  initial  status  parameters  are  equal.  Next,  a  model  is  estimated  where  the  change 
parameters  are  constricted  to  be  equal  between  groups.  Based  on  the  outcome  of  the  first  test,  the 
initial  status  parameter  is  left  constrained  or  unconstrained.  If  there  is  a  significant  drop  in  fit 
when  the  mean  change  estimate  is  constrained  to  be  equal  compared  to  when  the  mean  change 
estimate  is  unconstrained,  we  can  conclude  there  is  a  significant  difference  in  the  mean  change 
between  the  two  groups.  A  final  model  is  detennined  based  on  whether  the  initial  status  latent 
constructs  are  equivalent  and  whether  the  change  latent  constructs  are  equivalent  between 
groups. 


Stage  3.  In  order  to  examine  predictors  and  outcomes  of  change,  there  must  be  adequate 
amounts  of  variance  in  the  mean  change  factor.  No  variance  in  the  change  factor  indicates  that 
all  of  the  officers  in  the  sample  are  changing  along  similar  trajectories.  If  there  is  non-significant 
amount  of  variance  in  the  change  factor  then  no  additional  analyses  will  be  conducted.  To 
examine  key  predictors  and  outcomes  of  change,  the  predictor  variables  are  added  to  the  model 
by  regressing  the  change  factor  onto  the  predictor  variables  of  interest  (see  Figure  1,  Change  on 
RBI).  The  outcomes  are  added  to  the  model  by  regressing  the  outcome  variable  onto  the  change 
factor  (see  Figure  1,  Separation  on  Change).  Multiple  predictors  were  included  in  the  model  and 
each  predictor  was  examined  for  significance. 

Results 

Change  Over  Time 

The  following  describes  the  results  of  the  LGMs  assessing  linear  and  non-linear  change. 
Note,  mean  change,  listed  in  the  results,  represents  the  average  change  between  each  time  point. 
Table  4.2  lists  the  full  results  for  each  variable  including  the  goodness-of-fit  measures  for  each 
model. 


The  in-service  sample  demonstrated  a  significant  increase  in  Affective  Commitment  over 
time  while  the  enlistment  option  stayed  relatively  static  over  time  {Mean  change  =  .04;  see 
Figure  4.2).  On  average,  Affective  Commitment  increased  by  .12  or  3%  at  each  point  of 
measurement  for  the  in-service  sample. 
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Figure  4.2.  Latent  Growth  Model  of  Affective  Commitment  for  In-Service  and  Enlistment 
Option  Samples 

There  was  a  significant  non-linear  change  in  Continuance  Commitment  over  time  for  the 
enlistment  option  {Mean  change  =  -.11).  Specifically,  there  was  a  curvilinear  decrease  in 
commitment,  such  that  between  the  Beginning  of  OCS  and  the  end  of  BOLC  III/B  there  was  an 
increase  in  commitment;  however,  nearing  the  end  of  ADSO  there  was  a  decrease  in 
commitment.  There  was  no  significant  change  for  the  in-service  sample;  Continuance 
Commitment  remained  relatively  static  over  time  (see  Figure  4.3). 


Figure  4.3.  Latent  Growth  Model  of  Continuance  Commitment  for  In-Service  and  Enlistment 
Option  Samples 

Career  Intentions  for  both  the  in-service  {Mean  change  =  -.01)  and  enlistment  option 
{Mean  change  =  .01)  samples  were  relatively  static  over  time  (see  Figure  4.4). 
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Figure  4.4.  Latent  Growth  Model  of  Career  Intentions  for  In-Service  and  Enlistment  Option 
Samples 

Both  the  enlistment  option  and  in-service  samples  showed  a  significant  decrease  in 
Satisfaction  with  Branch  Assignment  over  time  (see  Figure  4.5).  On  average,  Satisfaction  with 
Branch  Assignment  decreased  by  .27  or  6%  at  each  point  of  measurement  for  both  samples. 


Figure  4.5.  Latent  Growth  Model  of  Satisfaction  with  Branch  Assignment  for  In-Service  and 
Enlistment  Option  Samples 
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Table  4.2.  Stage  1  Latent  Growth  Model  Results 


Mean  Mean 


Initial 

Linear 

Quadratic 

Scale 

Group 

N 

Model 

Status 

Change 

Change 

X2 

df 

CFI 

TLI 

RMSEA 

SRMR 

In-Service 

281 

Linear 

3.61 

.12 

- 

6.83 

5 

.99 

.99 

.04 

.05 

Affective 

Commitment 

Quadratic 

3.60 

.15 

-.01 

30.22 

4 

.88 

.82 

.15 

.14 

Enlistment 

Option 

208 

Linear 

3.62 

.04 

_ 

8.21 

6 

.99 

.99 

.04 

.07 

Quadratic 

3.58 

.14 

-.03 

19.83 

5 

.92 

.90 

.12 

.13 

In-Service 

204 

Linear 

2.87 

.05 

- 

7.67 

6 

.94 

.94 

.10 

.09 

Continuance 

Quadratic 

2.97 

-.14 

.06 

27.59 

4 

.88 

.82 

.17 

.12 

Enlistment 

Option 

148 

Linear 

2.73 

.01 

_ 

11.82 

5 

.96 

.95 

.10 

.05 

Commitment 

Quadratic 

2.55 

.35 

-.11 

2.56 

3 

1.00 

1.00 

.00 

.03 

In-Service 

342 

Linear 

3.54 

-.01 

— 

10.07 

5 

.99 

.99 

.05 

.10 

Career  Intentions 

Enlistment 

Option 

258 

Linear 

2.72 

.01 

3.02 

5 

1.00 

1.00 

.00 

.05 

Satisfaction  with 

In-Service 

301 

Linear 

4.27 

-.27 

14.23 

1 

.91 

.72 

.21 

.06 

Branch  Assignment 

Enlistment 

Option 

217 

Linear 

4.41 

-.27 

_ 

.92 

1 

1.00 

1.00 

.00 

.02 

Note.  Bolded  results  indicate  significance  at p<  .05.  CFI  =  Comparative  Fit  Index;  TLI  =  Tucker  Lewis  Fit  Index;  SRMR  =  Standardized  Root  Mean  Square  Residual;  RMSEA  = 
Root  Mean  Square  Error  of  Approximation;  CFI  and  TLI  values  >  .95  and  SRMR  and  RMSEA  values  <  .08  indicate  acceptable  fit.  Initial  status  represents  the  mean  starting  value 
of  the  variable  of  interest  across  all  individuals.  Change  represents  the  mean  change  at  each  point  of  measurement  in  the  variable  of  across  all  individuals.  Results  are  based  on 
FIML  likelihood  analyses  and  include  individuals  with  data  on  at  least  two  time  points. 


Differences  in  Changes  over  Time  by  Entry  Status 

Several  of  the  first  stage  LGMs  showed  differences  in  the  change  trajectory  for  the  in- 
service  and  enlistment  option  samples,  making  it  unnecessary  to  directly  test  the  differences  in 
mean  change  and  initial  status.  Specifically,  the  change  trajectory  for  Affective  Commitment  and 
Continuance  Commitment  differed;  the  in-service  sample  showed  significant  linear  change  on 
Affective  Commitment  while  the  enlistment  options  sample  showed  no  change.  Similarly,  the 
enlistment  option  sample  showed  a  significant  non-linear  change  on  continuance  commitment 
while  the  in-service  sample  showed  no  change.  We  directly  tested  for  difference  in  mean  change 
and  initial  status  for  Career  Intentions,  Morale,  and  Satisfaction  with  Branch.  Table  4.3  displays 
the  full  results  for  the  group  comparisons. 

•  For  Career  Intentions,  there  was  no  difference  in  change  between  the  in-service  and 
enlistment  option  samples.  However,  there  was  a  significant  difference  in  initial  status 
such  that  the  in-service  sample  showed  higher  initial  Career  Intentions  compared  to  the 
enlistment  option  sample.  Given  there  was  no  difference  in  change,  the  difference  in 
Career  Intentions  remained  consistent  over  time. 

•  There  was  no  significant  difference  in  initial  status  or  change  in  Morale. 

•  There  was  not  a  significant  difference  in  initial  status  or  change  for  Satisfaction  with 
Branch  Assignment.  Both  samples  had  similar  levels  of  satisfaction  with  their  branch 
assignment  at  the  End  of  OCS  and  both  samples’  satisfaction  decreased  over  time. 

Overall,  the  in-service  and  enlistment  option  samples  had  similar  changes  on  Branch 
Satisfaction — it  decreased  over  time.  Generally,  the  in-service  sample  planned  to  stay  with  the 
Anny  longer  than  the  enlistment  option  sample — this  did  not  change  over  time.  While  Career 
Intentions  did  not  change  for  the  in-service  sample,  Affective  Commitment  did  increase. 
Continuance  Commitment  initially  increased  for  the  enlistment  option  sample  but  decreased  as 
officers  neared  the  end  of  their  ADSO. 


Table  4.3.  Latent  Growth  Modeling  Result  Assessing  Difference  in  OCS  Accession  Option 


Model 

t 

df 

CFI 

TLI 

RMSEA 

SRMR 

A%2 

Adf 

Baseline 

13.09 

10 

1.00 

1.00 

.03 

.08 

Career  Intentions 

Equal  IS 

123.46 

11 

.86 

.85 

.19 

.19 

110. 37a 

1 

Equal  CH 

13.26 

6 

1.00 

1.00 

.03 

.08 

,17b 

1 

Baseline 

15.15 

2 

.94 

.83 

.16 

.05 

Satisfaction  with 

1.96a 

Branch  Assignment 

Equal  IS 

17.11 

3 

.94 

.88 

.14 

.05 

1 

Equal  IS-CH 

17.55 

4 

.94 

.91 

.11 

.05 

,44c 

1 

Baseline 

18.18 

4 

.89 

.83 

.13 

.25 

Morale 

Equal  IS 

21.97 

5 

.86 

.84 

.13 

.28 

3.79a 

1 

Equal  IS-CH 

9.42 

6 

.99 

.99 

.06 

.06 

-8.76c 

1 

Note.  Bolded  results  indicate  significance  at  p  <  .05.  Italicized  Models  indicate  the  model  that  was  determined  to  be  the  best 
fitting.  a Difference  between  the  baseline  model  and  the  Equal  IS  model.  b  Difference  between  the  Baseline  model  and  the  Equal 
CH  model. c  Difference  between  the  Equal  IS  model  and  the  Equal  IS-CH  model.  CFI  =  Comparative  Fit  Index;  TLI  =  Tucker 
Lewis  Fit  Index;  SRMR  =  Standardized  Root  Mean  Square  Residual;  RMSEA  =  Root  Mean  Square  Error  of  Approximation;  CFI 
and  TLI  values  >  .95  and  SRMR  and  RMSEA  values  <  .08  indicate  acceptable  fit. 
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Predictors  and  Outcomes  of  Change 


To  assess  predictors  and  outcomes  of  change,  there  must  be  non-zero  variance  in  the 
latent  variable  estimate  of  change.  To  assess  whether  the  change  variance  was  non-zero,  we 
examined  the  latent  variance  produced  by  the  model  and  the  associated  significance  test. 
Significant  variance  indicates  that  the  change  trajectory  differs  within  the  sample.  There  was 
significant  variance  in  the  change  factor  for  Morale  in  the  enlistment  option  and  Career 
Intentions  for  both  in-service  and  enlistment  option  samples.  While  average  change  in  Career 
Intentions  was  non-significant,  the  significant  variance  estimates  indicate  that  there  are 
differences  within  the  sample — there  may  be  some  individuals  with  static  Career  Intentions, 
others  with  increasing  Career  Intentions  and  others  with  decreasing  Career  Intentions.  Therefore, 
predictors  and  outcomes  will  be  examined  for  both  Morale  and  Career  Intentions. 

The  Core  RBI,  Core  Work  Values,  SJT  and  selection  composite  scales  were  examined  as 
predictors  of  change.  Table  4.4  lists  the  full  results  for  each  model  including  the  goodness-of-fit 
measures  and  the  multiple  correlations. 

•  Morale  -  The  Core  RBI  scales  and  the  Core  Work  Values  scales  were  significant 
predictors  of  change  in  Morale.  The  Core  RBI  and  Core  Work  Values  scales 
accounted  for  13%  and  20%  of  the  variance  in  change  in  Morale,  respectively. 

The  SJT  and  selection  composite  accounted  for  a  non-significant  portion  of  the 
variance.  The  latent  correlations  between  the  Core  RBI  scales  and  change  in 
Morale  showed  Hostility  to  Authority  (positively  related  to  change  in  Morale), 
Stress  Tolerance  (positive),  and  Self-Efficacy  (negative)  as  having  the  highest 
predictive  utility  of  change  in  Morale.  Altruism/Benevolence  (negative)  and 
Teamwork  (positive)  showed  the  highest  predictive  utility  of  change  in  Morale 
among  the  Core  Work  Value  scales. 

•  Career  Intentions  -  The  Core  RBI,  Core  Work  Values,  SJT  and  the  selection 
composite  predicted  non-significant  amounts  of  variance  in  change  in  Career 
Intentions  for  both  the  in-service  and  enlistment  option  sample. 

Career  Intentions  at  Time  4,  Separations,  and  Award  Perfonnance  were  examined  as 
outcomes  of  change.  Significant  relationships  are  described  in  text,  the  remaining  results  can  be 
found  in  Table  4.4. 

•  Morale  -  Stable  or  increasing  Morale  over  time  was  a  positive  predictor  of  Career 
Intentions  at  Time  4,  accounting  for  34%  of  the  variance.  Change  in  Morale  was 
also  a  negative  predictor  of  Separation,  accounted  for  16%  of  the  variance.  That 
is,  as  Morale  increases  over  time,  an  officer’s  intention  to  stay  in  the  Army 
increased  and  the  likelihood  of  separating  from  active  duty  officer  status 
decreased.  Change  in  Morale  over  time  was  not  a  significant  predictor  of  Awards. 
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Table  4.4.  Latent  Growth  Results  of  Predictors  and  Outcomes  of  Change 


n 

t 

df 

CFI 

TLI 

RMSEA 

SRMR 

R 

Morale  -  Enlistment  Option 

Predictors 

RBI 

185 

20.81 

12 

.90 

.75 

.06 

.06 

.37 

Work  Values 

185 

9.15 

10 

1.00 

1.00 

.00 

.04 

.45 

SJT 

185 

.76 

3 

1.00 

1.00 

.00 

.02 

.28 

Selection  Composite 
Outcomes 

185 

16.91 

10 

.92 

.80 

.06 

.06 

.27 

Career  Intentions 

185 

12.70 

5 

.93 

.91 

.09 

.10 

.58 

OMF:  Total  Awards 

185 

2.68 

5 

1.00 

1.00 

.00 

.04 

.12 

OMF:  Weight  Awards 

185 

2.42 

5 

1.00 

1.00 

.00 

.04 

.06 

Separations 

185 

1.67 

5 

1.00 

1.00 

.00 

- 

.40 

Career  Intentions  -  Enlistment  Option 

Predictors 

RBI 

258 

32.80 

24 

.98 

.97 

.04 

.03 

.37 

Work  Values 

258 

24.38 

20 

.99 

.98 

.03 

.04 

.50 

SJT 

258 

8.32 

8 

1.00 

1.00 

.01 

.06 

.28 

Selection  Composite 
Outcomes 

258 

27.71 

20 

.98 

.97 

.04 

.04 

.38 

OMF:  Total  Awards 

258 

3.96 

8 

1.00 

1.00 

.00 

.04 

.10 

OMF:  Weight  Awards 

258 

3.68 

8 

1.00 

1.00 

.00 

.04 

.06 

Career  Intentions  -  In-Service 

Predictors 

RBI 

342 

41.44 

24 

.97 

.94 

.05 

.06 

.53 

Work  Values 

342 

42.22 

20 

.95 

.92 

.06 

.07 

.51 

SJT 

342 

19.89 

8 

.97 

.97 

.07 

.14 

.46 

Selection  Composite 

342 

41.42 

20 

.95 

.92 

.06 

.07 

.50 

Note.  Bolded  values  indicate  significance  at  p  <  .05.  CFI  =  Comparative  Fit  Index;  TLI  =  Tucker  Lewis  Fit  Index;  SRMR  = 
Standardized  Root  Mean  Square  Residual;  RMSEA  =  Root  Mean  Square  Error  of  Approximation;  CFI  and  TLI  values  >  .95  and 
SRMR  and  RMSEA  values  <  .08  indicate  acceptable  fit. 


Discussion 


Longitudinal  Change 

A  number  of  variables  showed  significant  change  over  time.  While  there  is  no  direct 
indication  of  why  Satisfaction  with  Branch  Assignment  decreased  over  time  in  both  samples,  we 
propose  one  theory.  For  the  majority  of  the  respondents,  branch  selections/assignments  were 
made  at  the  midpoint  (end  of  week  six)  of  OCS.  The  decrease  in  branch  satisfaction  coincides,  in 
time,  with  branch  selection  (End  of  OCS),  branch  training  (BOLC  1 1  I  B),  and  finally  their  in-unit 
branch  assignment.  These  results  suggest  that  the  perception  and  the  reality  of  branching — from 
OCS,  through  BOLC  III/B,  and  into  the  unit — are  very  different.  What  officers  conceived  or 
expected  (as  the  ideal)  may  be  very  different  from  what  they  actually  experienced,  thus 
accounting  for  a  decrease  in  satisfaction  over  time.  Providing  Officers  with  a  realistic  preview  of 
branch-specific  duties  in  OCS  may  help  to  reduce  the  decrease  in  Satisfaction  with  Branch 
Assignment  (Oliver,  Ardison  et  ah,  201 1;  Oliver  et  ah,  2012).  Additional  research  should  be 
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conducted  to  examine  the  role  of  branch  preference,  assignment,  and  other  external  factors  such 
as  deployments  in  the  decrease  in  Satisfaction  over  time. 

Group  Differences 

While,  in  general,  there  were  few  attitudinal  measure  differences  between  the  in-service 
and  enlistment  option  samples,  the  most  striking  difference  was  with  Career  Continuance 
Intentions.  The  in-service  officers  had  much  higher  Career  Continuance  Intention  scores  than  the 
enlistment  option  group,  and  this  difference  remained  consistent  over  time.  The  in-service  group 
is  older,  has  more  time  in  service,  and  is  closer  to  retirement  age.  This  may  explain  their  higher 
career  intention  ratings.  On  the  other  hand,  the  enlistment  option  group  is  generally  just  out  of 
college  and  younger,  and  thus  may  not  yet  have  made  a  final  career  decision.  Additionally, 
scores  on  Affective  Commitment  appear  to  be  higher  for  in-service  than  the  enlistment  option,  at 
least  once  within  the  in-unit  environment.  This  may  also  relate  to  their  prior  military  history  as 
enlisted  Soldiers,  because  they  have  already  invested  more  time  in  the  Anny. 

Predictors  and  Outcomes 

We  found  very  little  variance  among  officers  in  the  mean  change  estimate.  That  is,  for  the 
in-service  and  enlistment  options  the  change  functions  for  the  attitudinal  variables  were  similar 
for  most  officers.  With  the  exception  of  Morale  and  Career  Intentions  there  were  no  significant 
individual  differences  in  how  officers  changed  over  time.  The  lack  of  variance  suggests  that  there 
may  be  a  group  effect  that  is  driving  how  Affective  Commitment,  Continuance  Commitment, 
and  Branch  Satisfaction  change  over  time. 

The  Core  RBI  and  Core  Work  Values  scales  were  significant  predictors  of  change  in 
Morale.  In  terms  of  implications  for  selecting  in-service  and  enlistment  option  candidates  into 
OCS,  this  is  the  most  fruitful  finding  in  that  both  the  RBI  and  Work  Values  scales  showed 
promise  to  explain  change  in  certain  attitudes,  such  as  Morale.  In  turn,  change  in  Morale 
predicted  Career  Intentions  and  Separations.  Specifically,  those  with  positive  changes  in  Morale 
tended  to  have  higher  career  aspirations  and  lower  rates  of  separating.  These  findings  suggest 
that  the  RBI  and  Work  Values  could  be  useful  in  selecting  individuals  with  a  higher  propensity 
for  positive  change  in  Morale  over  time,  which  in  turn  may  lead  to  a  higher  likelihood  that  these 
individuals  stay  in  the  Army.  Alternatively,  the  predictors  could  be  used  to  select  out  individuals 
with  a  higher  propensity  of  decreased  Morale  over  time,  which  would  reduce  the  number  of 
individuals  with  a  high  probability  of  having  shorter  careers  with  the  Anny. 

Conclusion 

This  research  offers  a  perspective  on  how  attitudes  change  over  time  in  the  first  three 
years  of  active  duty  service  for  officers  and  examines  potential  predictors  and  outcomes  of 
change.  These  findings  provide  initial  groundwork  for  understanding  how  attitudes  change.  As 
large  sample  sizes  become  available,  more  research  needs  to  be  conducted  to  determine  the 
generalizability  of  these  findings.  Initial  findings  suggest  that  there  may  be  some  utility  in  using 
these  findings  to  guide  selection  decisions,  but  more  research  needs  to  be  conducted  to  better 
understand  the  impact  of  changing  attitudes  on  performance  and  continuance  in  the  Army. 
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CHAPTER  5:  CONCLUSIONS  AND  NEXT  STEPS 


Teresa  L.  Russell  and  Matthew  T.  Allen 
Human  Resources  Research  Organization 

Mark  C.  Young 
U.S.  Army  Research  Institute 

The  overarching  purpose  of  the  ongoing  research  program  described  in  this  report  is  to 
develop  a  valid  assessment  for  selecting  candidates  -  whether  through  the  in-service  or 
enlistment  option  -  into  the  U.S.  Anny’s  Officer  Candidate  School  (OCS).  To  accomplish  this, 
ARI  and  HumRRO  developed  and  validated  a  battery  of  measures  called  the  Officer  Background 
and  Experiences  Fonn  (OBEF)  in  2008  (see  Russell  &  Tremble,  2011).  The  OBEF  assesses  non- 
cognitive  aspects  of  a  candidate’s  persona,  such  as  personality/temperament,  values,  and 
judgment.  This  compliments  the  Army’s  current  assessment  tool — the  Anned  Services 
Vocational  Aptitude  Battery  (ASVAB) — which  assesses  mostly  knowledge  and  cognitive 
aptitudes. 

The  purpose  of  the  present  report  was  to  expand  on  the  initial  work  performed  in  2008  by 
detennining  whether  the  OBEF  could  predict  OCS-commissioned  officer  perfonnance  and 
continuance  beyond  the  training  environment.  Specifically,  the  objectives  of  the  present  research 
were  to  (a)  collect  longer-term  performance  and  continuance  measures,  (b)  validate  the  OBEF 
against  these  outcomes,  (c)  examine  changes  in  officer  attitudes  over  time,  and  (d)  determine  the 
antecedents  and  consequences  of  those  attitude  changes.  The  research  team  faced  a  number  of 
challenges  in  accomplishing  these  objectives,  and  learned  a  number  of  lessons  that  can  be 
applied  to  future  longitudinal  validation  studies. 

Summary  of  Methodological  Approach 

Our  research  approach  was  driven  by  an  urgency  to  obtain  empirically  supported 
evidence  to  infonn  near-tenn  recommendations  for  enhancing  the  Anny’s  OCS  selection 
process.  This  urgency  was  driven  by  the  desire  to  capture  key  criteria  of  interest  from  our  sample 
prior  to  their  separation  from  active  duty.  However,  long-tenn  career  outcome  measures  (e.g., 
performance  as  a  mid-career  officer) — which  would  have  been  most  desirable  for  use  in  the  test 
validation — were  unavailable  for  our  relatively  young  2008  OCS  cohort.  Accordingly, 
considerable  effort  was  focused  on  obtaining  the  most  relevant  and  viable  early  career  outcome 
measures  that  could  be  used  as  “interim”  criteria.  We  used  innovative  approaches  to  obtain  the 
richest  and  most  complete  set  of  emerging  outcome  measures  feasible,  given  the  short  timeframe 
for  this  project.  These  efforts  involved  the  use  of  multiple  data  collection  approaches  and  we 
were  successful  in  obtaining  a  relatively  wide  range  of  officer  perfonnance  outcome  measures. 

In  any  context  obtaining  criterion  data  for  a  sample  approximately  three  years  after  the 
original  assessment  date  is  a  logistical  challenge.  This  is  particularly  true  in  the  present  research, 
as  the  2008  OCS  cohort  is  now  dispersed  around  the  globe.  Concerns  about  privacy,  inconsistent 
administrative  records,  obtaining  adequate  response  rates,  and  the  desire  to  collect  infonnation 
on  a  broad  range  of  perfonnance  and  retention-related  criteria  all  increased  the  complexity  of  the 
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data  collection  process.  With  these  challenges  in  mind,  the  research  team  took  a  number  of  steps 
to  ensure  that  the  data  were  of  high  enough  quality  to  answer  key  research  questions  of  interest 
and  mitigate  logistical  difficulties.  These  steps  included: 

•  Developing  a  self-report  criterion  instrument  that  combined  verifiable  performance 
questions,  subjective  performance  questions,  and  attitudinal proxy  measures.  The 
“In-Unit  Survey”  developed  for  the  present  research  leveraged  previous  ARI- 
sponsored  research  in  a  number  of  ways.  First,  the  instrument  relied  heavily  on 
verifiable  perfonnance  and  retention  questions,  which  previous  research  has  shown  to 
be  relatively  accurate  (Campbell  &  Knapp,  2001).  Second,  in  key  performance 
domains  where  verifiable  performance  data  was  not  available,  subjective  ratings  were 
gathered.  However,  by  using  a  relative  perfonnance  scale  (as  opposed  to  absolute 
rating  scales;  see  items  39  through  44  in  Appendix  B),  we  hoped  these  subjective 
ratings  would  be  less  influenced  by  social  desirability  than  traditional  behavioral 
rating  scales.  Finally,  attitudinal  scales  that  recent  research  has  found  to  be  reasonable 
proxies  of  key  outcomes  of  interest  (such  as  retention  beyond  their  service  obligation) 
were  also  administered  (Oliver,  Ingerick  et  al.,  2012).  In  combination,  these  items 
provided  a  wide-ranging  (though  not  complete,  see  Campbell,  2012)  picture  of  the 
perfonnance  and  retention-related  factors  critical  to  validating  the  OBEF. 

•  Taking  steps  to  maximize  the  response  rate.  Traditionally,  the  response  rate  for 
unproctored  internet  surveys  in  the  Army  is  quite  low.  To  enhance  the  In-Unit  Survey 
response  rate,  the  researchers  (a)  promised  a  short  survey  (approximately  10  minutes) 
in  the  solicitation  email,  (b)  sent  multiple  reminders  to  complete  the  survey,  and  (c) 
asked  the  fonner  Commandant  of  OCS  to  endorse  the  solicitation  email.  We  feel  that 
all  of  these  factors  contributed  to  a  higher-than-typical  response  rate  (exceeding  40%) 
in  a  short  amount  of  time. 

•  Piloting  new  methods  for  collecting  supervisory  ratings.  At  the  time  of  this  writing, 
the  research  team  was  also  piloting  a  non-obtrusive  approach  for  gathering 
supervisory  ratings  of  in-unit  perfonnance  among  the  2008  OCS  officer  cohort.  Early 
returns  on  the  new  approach  currently  being  tested  (see  Appendix  B  for  more  details) 
were  encouraging. 

•  Mining  administrative  records.  To  the  extent  possible,  the  research  team  relied  on 
administrative  personnel  records  to  obtain  perfonnance  and  retention  information 
about  the  participants  in  the  present  sample.  When  feasible,  we  also  obtained  these 
records  concurrently  from  multiple  data  sources  (i.e.,  military  agencies).  The  rationale 
was  to  use  the  strengths  of  some  personnel  files  to  help  offset  the  weaknesses  in 
others.  This  approach  was  used  in  developing  the  integrated  personnel  record 
database  for  constructing  our  administrative  criteria.  In  combination,  all  of  these 
strategies  helped  to  increase  the  quality  and  completeness  of  the  data,  thereby 
enhancing  the  available  sample  size  for  many  of  the  validation  analyses  reported  here. 

•  Applying  state-of-the-art  missing  data  methods  in  conducting  analyses.  Collecting 
data  longitudinally  at  multiple  points  in  time  can  lead  to  small  samples  of  respondents 
with  data  at  all  points.  Traditional  methods  for  handling  these  missing  data,  such  as 
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listwise  deletion  (i.e.,  omitting  those  with  missing  data  at  any  one  time  point  from  the 
analysis),  can  lead  to  low  power  and  biased  statistical  estimates  (Enders  &  Bandalos, 
2001;  see  Chapter  3  for  more  details).  To  account  for  this  limitation,  we  used  state-of- 
the-art  missing  data  techniques  in  our  analysis.  Previous  research  has  shown  these 
techniques  to  be  more  accurate  than  traditional  methods. 

•  Applying  corrections  for  statistical  artifacts.  To  account  for  potential  artifacts  in  the 
data,  the  validation  analyses  applied  statistical  corrections  for  range  restriction  and 
shrinkage.  These  estimates  are  a  more  conservative  test  of  the  efficacy  of  the  OBEF 
for  predicting  key  performance  and  retention  outcomes  by  giving  more  weight  to  the 
Armed  Forces  Qualification  Test  (AFQT)  in  the  analyses  and  penalizing  the  OBEF 
for  contributing  multiple  scales  to  the  regression  equation. 

Variants  of  many  of  these  procedures  have  been  applied  in  previous  research.  However, 
we  feel  this  combination  of  steps  will  be  useful  to  other  researchers,  particularly  military 
researchers,  who  want  to  collect  and  analyze  empirical  data  without  relying  heavily  on  face-to- 
face  data  collections. 


Summary  of  Results 

One  key  purpose  of  this  research  was  to  validate  the  OBEF  against  performance-related, 
retention-related,  and  attitudinal  outcomes  for  both  the  enlistment  option  and  in-service  samples. 
In  support  of  this  objective,  we  captured  a  broad  spectrum  of  the  officer  criterion  space  through 
the  theory- informed  development  of  14  different  scales  built  from  survey  data  and  administrative 
records.  The  overall  pattern  of  results  clearly  showed  that  the  OBEF  was  predictive  of  important 
officer  career  outcomes  roughly  three  years  after  receiving  their  commission.  Significantly,  some 
degree  of  validity  evidence  was  found  for  nearly  all  of  the  criterion  measures  we  examined. 

From  the  analyses  conducted  in  Chapter  3,  we  can  further  conclude  the  following: 

1 .  The  OBEF  demonstrated  significant  promise  for  predicting  key  in-unit  performance, 
continuance,  and  attitudinal  outcomes  in  the  enlistment  option  sample.  Specifically, 
temperament  (measured  using  the  Core  Rational  Biodata  Inventory  [RBI])  and  work 
values  (measured  using  the  Core  Work  Values  scales)  aspects  of  the  OBEF  were 
particularly  strong  predictors  of  officer  performance,  attitudes,  and  continuance.  The 
Situational  Judgment  Test  (SJT)  also  predicted  non-trivial  variance  in  certain  criteria 
beyond  the  AFQT  (most  notably  In-Unit  Career  Continuance  Intentions  and 
Commitment).  All  three  measures  were  predictive  of  officer  commitment  to  the  Anny 
while  in  their  units,  but  generally  less  predictive  of  leadership  and  personal  discipline. 

2.  The  OBEF  also  demonstrated  promise  for  predicting  key  in-unit  performance, 
continuance,  and  attitudinal  outcomes  in  the  in-service  sample.  As  with  the  enlistment 
option  sample,  the  OBEF  was  significantly  predictive  of  key  outcomes  such  as  affective 
commitment,  technical  knowledge,  branch  satisfaction,  morale,  and  recycles  during 
Initial  Officer  Training  (IOT)  beyond  the  AFQT.  The  Core  RBI  scales  were  particularly 
strong  predictors  of  these  outcomes,  though  the  Core  Work  Values  scales  were  also 
predictive  of  a  number  of  outcomes.  The  SJT  was  generally  not  a  significant  predictor  of 
any  outcomes  in  this  sample. 
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3.  Some  OBEF  scales  consistently  predicted  aspects  of  both  in-service  and  enlistment 
option  officer  outcomes,  while  other  scales  were  more  predictive  in  one  sample  than 
another.  The  Core  Work  Values  Rugged  Leadership  and  Teamwork,  and  the  Core  RBI 
Peer  Leadership,  Self-Efficacy,  and  Anny  Affective  Commitment  scales  predicted 
outcomes  in  both  samples.  In  contrast,  the  Skill  Development  Work  Values  was  a 
significant  predictor  of  multiple  outcomes  in  the  in-service  sample,  but  was  non¬ 
significant  across  most  outcomes  in  the  enlistment  option  sample.  RBI  Fitness 
Motivation  predicted  physical  fitness  and  no  other  outcomes  in  the  in-service  sample,  but 
predicted  multiple  outcomes  (e.g.,  leadership,  commitment)  in  the  enlistment  option 
sample.  RBI  Stress  Tolerance  also  predicted  multiple  outcomes  in  the  enlistment  option 
sample,  but  not  the  in-service  sample. 

4.  The  OBEF  composites  developed  in  2008 for  selecting  in-service  and  enlistment  option 
officers  into  OCS predicted  key  performance  and  continuance  outcomes  three  years 
later.  Specifically,  the  in-service  composite  predicted  key  attitudinal  (commitment, 
branch  satisfaction,  morale)  and  performance  (physical  fitness)  outcomes,  while  the 
enlistment  option  composite  predicted  commitment,  physical  fitness,  completion  of 
initial  officer  training,  and  active  duty  separation. 

A  secondary  purpose  of  this  research  was  to  examine  changes  in  OCS-commissioned 
officer  attitudes  over  time,  the  factors  that  predict  attitude  change,  and  the  consequences  of 
changing  attitude  for  outcomes  near  the  end  of  their  initial  Active  Duty  Service  Obligation 
(ADSO).  From  the  analyses  conducted  in  Chapter  4,  we  can  conclude  the  following: 

1 .  In-Service  and  enlistment  option  officers  differed  in  their  change  trajectories  for  multiple 
attitudinal  outcomes.  In-service  officer  commitment  and  career  intentions  stayed 
relatively  stable  over  time.  The  results  were  more  mixed  for  the  enlistment  option 
sample,  showing  stable  affective  commitment  and  career  intentions,  but  decreasing 
continuance  commitment,  over  time.  Both  samples  showed  decreased  branch  satisfaction 
over  time.  Officers  in  the  in-service  sample  generally  had  longer  career  intentions  than 
their  enlistment  option  counterparts. 

2.  Change  in  Morale  was  predicted  by  the  OBEF  and  linked  to  officer  separation. 
Specifically,  the  Core  RBI  Hostility  to  Authority,  Self-Efficacy,  and  Continuance 
Commitment  scales,  and  the  Core  Work  Values  Altruism/Benevolence  and  Teamwork 
scales  predicted  change  in  Morale.  Change  in  Morale  was  predictive  of  both  In-Unit 
Career  Intentions  and  Separations  from  active  duty  service,  such  that  stable  or  increasing 
change  in  Morale  decreased  the  likelihood  of  leaving  in  the  Anny. 

3.  Self-reported  Career  Intentions  are  related  to  actual  separations .  Officers  were  asked  to 
describe  their  career  intentions  at  four  points  in  time — at  the  Beginning  of  OCS(Time  1), 
at  the  End  of  OCS(Time  2),  during  BOLC  III/B  (Time  3),  and  while  In-Unit  (Time  4), 
nearing  completion  of  their  ADSOs.  Regardless  of  the  timing,  career  intentions  were 
correlated  with  each  other  significantly  in  both  the  in-service  and  enlistment  option 
samples — for  example.  Time  1  correlated  with  Time  2  (r  =  .68  and  .66,  respectively); 
Time  1  correlated  with  Time  4  (r=  .42  and  .45,  respectively).  Time  2,  3,  and  4  career 
intentions  each  correlated  significantly  with  actual  separations,  albeit  the  magnitude  of 
the  correlation  was  not  large  (see  Appendix  D).  Since  some  officers  who  intend  to 
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separate  may  not  have  yet  done  so,  we  expect  the  correlation  between  career  intentions 
and  actual  separation  to  increase  over  time.  Based  on  the  current  data,  career  intentions 
appear  to  be  a  moderate  predictor  of  actual  separation. 

Several  important  conclusions  can  be  drawn  from  the  results.  First,  the  OBEF  predicts 
key  early-career  perfonnance  and  retention  outcomes  well  beyond  the  point  of  initial  officer 
training.  Earlier  research  has  focused  only  on  criteria  that  could  be  collected  at  the  end  of  OCS 
such  as  the  OCS  Order  of  Merit  List  (OML)  ranking,  peer  ratings,  and  physical  and  academic 
performance  in  OCS.  Finding  that  the  OBEF  predicts  in-unit  outcomes  for  junior  officers  is  an 
important  contribution  and  a  milestone  in  the  ongoing  OBEF  validation  process.  Also,  the 
procedures  and  resources  used  to  collect  survey  and  administrative  data  for  this  report  will  be 
important  for  future  longitudinal  validation  efforts. 

Finally,  we  found  that  officer  attitudes,  such  as  satisfaction  with  their  branch  assignment, 
sometimes  changed  negatively  over  time.  It  is  tempting  to  suggest  that  the  “new”  might  have 
worn  off  for  the  young  trainees  as  they  joined  units;  however,  attitude  fonnation  is  a  complex 
process.  Societal  factors,  family  matters,  and  deployments  also  affect  attitudes.  For  example, 
surveys  of  officers  suggested  that  morale  of  officers  deployed  to  Iraq  increased  between  2009 
and  2011  while  it  decreased  for  those  deployed  to  Afghanistan,  and  morale  was  higher  for 
officers  in  garrison  than  on  deployment  (Riley,  Hatfield,  Nicely,  Keller-Glaze,  &  Steele,  2011). 
Changes  in  attitudes  for  the  2008  cohort  are  likely  tied  to  a  number  of  external  factors  that  are 
beyond  the  scope  of  the  current  paper. 


Limitations 

In  longitudinal  research,  the  timing  of  data  collection  at  key  milestones  is  critical.  To 
evaluate  the  potential  of  the  OBEF  for  predicting  early  career  outcomes,  we  collected  and 
analyzed  administrative  data  in  April  of  2012,  a  date  selected  because,  at  that  point,  our  2008 
sample  had  reached  the  end  of  their  ADSOs.  Separations  were  fairly  low,  from  a  data  analysis 
standpoint,  making  accurate  predictions  difficult.  Officers  do  not  have  to  re-enlist  at  the  end  of 
their  ADSOs  and  can  simply  opt  to  stay  for  an  unspecified  amount  of  time.  Therefore,  we  believe 
that  separations  for  this  sample  will  likely  continue  before  stabilizing.  Indeed,  some  of  the 
officers  who  told  us  in  the  In-Unit  survey  that  they  planned  to  leave  Active  Duty  had  not  done  so 
at  the  time  when  we  obtained  the  administrative  data.  Collecting  administrative  data  again  in  6 
months  or  one  year  would  allow  the  creation  of  more  mature  continuance  criteria  with  greater 
variance.  Such  criteria  would  likely  be  superior  to  existing  criteria  for  the  purpose  of  validating 
the  OBEF  against  continuance  behavior. 

Another  limitation  has  to  do  with  the  generalizability  of  research  results  to  the 
operational  setting.  Self-report  instruments  like  the  OBEF  are  often  criticized  for  their  potential 
for  response  distortion.  Response  distortion  occurs  when  a  respondent,  intentionally  or  not, 
inaccurately  presents  himself  in  a  positive  light.  There  is  more  pressure  on  the  applicant  to  distort 
responses  when  something  is  at  stake,  such  as  selection  into  OCS.  When  the  2008  sample  took 
the  OBEF  under  for-research-only  conditions,  they  had  no  external  pressure  toward  response 
distortion.  The  results  for  some  of  the  OBEF  scales  might  change  under  operational  conditions. 
Accordingly,  future  research  needs  to  evaluate  the  OBEF  under  such  conditions. 
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Future  Directions 


Future  OBEF  research  needs  to  focus  on  (a)  continued  examination  of  longitudinal 
outcomes  for  cohorts  of  OCS  officers  who  have  taken  the  OBEF  and  (b)  the  efficacy  of  the 
OBEF  for  use  in  an  operational  setting. 

Longitudinal  Database  Development  and  Validation 

Future  analyses  of  longitudinal  data  will  evaluate  the  extent  to  which  the  OBEF  results 
described  in  this  report  hold  up  over  time  and  across  samples.  In  all,  the  OBEF  has  been 
administered  to  three  cohorts  of  OCS  candidates.  The  current  report  discusses  longitudinal 
results  for  the  first  cohort,  tested  in  2008.  Two  additional  cohorts  were  tested,  one  in  2010  (5 
classes;  total  n  =  781;  Russell,  Allen,  &  Babin,  2011)  and  another  in  201 1  (5  classes;  total  n  = 
459;  Allen  et  ah,  2012). 

Overall,  the  basic  research  design  has  remained  constant  across  cohorts.  That  is,  the 
OBEF  was  administered  at  the  Beginning  of  OCS  (Time  1).  As  shown  in  Table  5.1,  at  Time  2, 
officer  candidates  took  the  End  of  OCS  survey  just  before  they  graduated  from  OCS.  Then, 
about  6  months  after  graduating  from  OCS,  the  participants  (then  second  lieutenants)  took  the 
BOLC  III/B  survey  (about  the  time  they  were  in  or  had  completed  BOLC  III/B).  While  Time  1, 
2,  and  3  data  have  been  collected  for  all  cohorts,  the  Time  4  In-Unit  Survey  has  only  been 
administered  to  the  2008  cohort.  Additionally,  separation  data  will  continue  to  mature  for  the 
2008  cohort  as  well  as  the  other  cohorts.  Furthermore,  promotions  to  captain  have  recently 
become  more  competitive,  thus  days  to  promotion  to  captain  will  be  a  criterion  of  interest  in 
coming  years. 


Table  5.1.  Longitudinal  Milestones  for  the  2008,  2010,  and  2011  OCS  Cohorts 


Cohort 

Milestones 

2008 

2010 

2011 

Time  1  -  Beginning  of  OCS  OBEF 
Time  2  -  End  of  OCS  Survey/Data 
Time  3  -  BOLC  III/B  Survey 

Time  4  -  In-Unit  Survey 

Complete  ADSO  (Separation  Data) 
Promotion  to  Captain 

V  6/08  -  12/08 

V  8/08  -  3/09 

V  2/09  -  8/09 

V  6/11  -9/11 

V  8/11  -3/12 
8/12-3/13 

V  2/10 -5/10 

V  4/10  -  7/10 

V  9/10  -  12/10 
9/12-  12/12 

4/13-7/13 

4/14  -  7/14 

V  5/11  -8/11 

V  7/11  -  12/11 

V  1/12-4/12 
12/13-3/14 

7/14-  12/14 

7/15  -  12/15 

Note.  V  indicates  data  collections  that  have  taken  place. 


Longitudinal  data  are  challenging  to  obtain,  and  if  the  emergent  data  are  not  collected 
during  particular  time  frames,  the  opportunity  for  capturing  them  can  be  lost.  ARI  has  made  a 
significant  investment  in  OCS  data  collections;  completing  the  databases  with  longitudinal  data 
must  be  done  in  a  timely  way  in  order  to  maximize  the  usefulness  and  potential  value  of  this 
investment. 
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The  Efficacy  of  the  OBEF  for  Operational  Use 

Eventually,  the  predictive  validity  of  the  OBEF  must  be  evaluated  with  applicants  in  an 
operational  setting.  While  the  OBEF  has  been  tested  extensively  in  ARI’s  OCS  selection 
research  program,  it  has  never  been  administered  to  an  applicant  population.  Research 
participants  were  officer  candidates  in  OCS  who  were  already  selected  on  the  basis  of  their 
ASVAB  test  scores,  academic  record,  and  perfonnance  before  a  selection  board.  The  OBEF  data 
were  collected  for  research  purposes  and  officer  candidates  were  assured  that  their  data  would 
not  be  used  by  the  Anny  for  any  other  purpose. 

As  of  this  writing,  plans  are  currently  underway  for  ARI  to  test  the  OBEF  in  an  applicant 
setting.  In  this  proposed  Initial  Operational  Test  and  Evaluation  (IOT&E),  the  Anny  would 
administer  a  shortened  version  of  the  OBEF  to  OCS  enlistment  option  applicants,  most  likely  at 
the  Military  Entrance  Processing  Stations  (MEPS).  Such  research  would  address  the 
generalizability  concerns  expressed  above  and  thus  is  a  logical  extension  of  the  current  research. 

The  IOT&E  would  allow  ARI  to  evaluate  the  potential  effects  of  response  distortion  and 
the  validity  of  the  OBEF  under  operational  conditions.  It  would  also  provide  data  that  could  be 
used  to  finalize  composites,  evaluate  those  composites,  and  determine  cut  scores.  To  date,  our 
research  has  focused  on  the  development  of  composites  for  predicting  perfonnance  and 
continuance,  sometimes  using  one  composite  to  predict  both  (Russell  &  Tremble,  2011)  and 
developing  multiple  composites  in  others  (Allen  et  ah,  2012).  However,  in  an  operational  setting, 
the  composite  scores  will  need  to  be  used  together  to  make  a  yes  or  no  decision  about  the 
selection  of  an  applicant.  Composite  scores  can  be  combined  in  a  compensatory  way  (such  that 
good  performance  on  one  makes  up  for  poor  perfonnance  on  the  other)  or  in  a  non-compensatory 
fashion  (where  there  are  separate  cut  scores  for  each  composite).  Also,  scores  can  be  used  to 
either  select  in  the  best  or  select  out  the  weakest  applicants.  Data  from  the  operational 
environment  can  be  used  to  simulate  the  pass  rates,  adverse  impact,  reliability,  and  validity  of 
alternative  scoring  systems.  Moreover,  future  research  should  not  only  select  the  scales  to  be 
used  in  composites  but  also  evaluate  the  validity  and  practicability  (in  tenns  of  applicant  flow 
rates)  of  alternative  scoring  and  cut  score  approaches.  Efforts  to  accomplish  these  objectives  are 
ongoing. 
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APPENDIX  A 


CORE  RBI  AND  WORK  VALUES  COMPONENT  SCALE  DESCRIPTIONS 


A-l 


Table  A.  1.  Description  of  Core  Rational  Biodata  Inventory  (RBI)  Component  Scales 

Scale  Definition 

Peer  Leadership  Seeks  positions  of  authority  and  influence.  Comfortable  with  being  in 

charge  of  a  group.  Willing  to  make  tough  decisions  and  accept 
responsibility  for  the  group’s  performance. 

Achievement  The  willingness  to  give  one’s  best  effort  and  to  work  hard  towards 

achieving  difficult  objectives. 

Fitness  Motivation  Degree  of  enjoyment  from  participating  in  physical  exercise.  Willingness  to 

put  in  the  time  and  effort  to  maintain  good  physical  conditioning. 

Stress  Tolerance  Ability  to  maintain  one’s  composure  under  pressure.  Remaining  calm  and 

in  control  of  one’s  emotions  instead  of  feeling  anxious  and  worried. 

Hostility  to  Authority  Being  suspicious  of  the  motives  and  actions  of  legitimate  authority  figures. 

Viewing  rules,  regulations,  and  directives  from  higher  authority  as  punitive 
and  illegitimate. 

Self-Efficacy  Feeling  that  one  has  successfully  overcome  work  obstacles  in  the  past  and 

that  one  will  continue  to  do  so  in  the  future. 

Traditional  Values  Acceptance  of  societal  values,  authority,  and  the  value  of  discipline. 

Army  Identification  The  degree  of  personal  identification  with,  and  intrinsic  interest  in 

becoming,  a  U.S.  Army  Soldier. 

Continuance  Commitment  The  degree  of  commitment  to  the  Army  because  alternatives  are  less 

attractive  or  because  the  cost  of  leaving  the  Army  is  too  great. 

Note.  The  information  in  this  table  was  taken  from  Russell  and  Tremble  (20 1 1 ).  A  validity  scale,  administered  to  detect  and 

adjust  for  inaccurate  responding,  was  also  administered  as  part  of  the  RBI. 

Table  A.  2.  Description  of  Core  Work  Values  Component  Scales 

Scale  Definition 

Rugged  Leadership  Preference  for  a  work  environment  in  which  one  is  responsible  for  leading 

others  under  physically  challenging  circumstances 

Teamwork  Preference  for  a  work  environment  in  which  one  can  establish  close  ties  with 

co-workers  and  work  as  part  of  a  team 

Flexibility/Choice  Preference  for  a  flexible  work  environment  that  respects  one’s  time  and 

personal  needs  and  in  which  one  can  work  independently 

Job  Security  Preference  for  a  work  environment  that  offers  job  security 

Structure/Recognition  Preference  for  a  highly  structured  work  environment  in  which  one  is 

recognized  and  admired  by  others 

Altruism/Benevolence  Preference  for  a  work  environment  in  which  one  can  help  others 

Skill  Development  Preference  for  a  work  environment  that  makes  full  use  of  one’s  skills  and 

abilities  and  offers  opportunities  for  learning  and  developing  new  skills 

Note.  The  information  in  this  table  was  taken  from  Russell  and  Tremble  (20 11). 
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APPENDIX  B 


IN-UNIT  (TIME  4)  SURVEY  CONTENT 
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Background 


1 .  What  is  your  Army  component? 

o  Regular  (Active)  Army 
o  Anny  Reserve 
o  Anny  National  Guard 

2.  What  is  your  current  pay  grade? 

o  01 
o  02 
o  03 

3.  What  is  your  marital  status? 
o  Married 

o  Legally  separated  or  filing  for  divorce 
o  Single,  never  manied 
o  Divorced 
o  Widowed 

4.  How  many  dependent  children  do  you  have  (for  whom  you  provide  over  half  of  their 
support)? 

o  None 
o  1 

o  2 

o  3  or  more 


Civilian  and  Military  Education/Training 

5.  What  is  the  highest  level  of  civilian  education  you  have  completed? 

o  Some  college  credits,  no  degree 
o  Associate  or  other  two-year  degree 
o  Bachelor's  degree 

o  Some  graduate  school  credits,  no  degree 

o  Master's  degree  (e.g.,  MA,  MS,  or  MBA) 

o  Doctorate  (PhD)  or  professional  degree  (e.g.,  MD,  DDS,  or  JD) 

6.  What  was  (were)  your  major  area(s)  of  study  as  a  college  undergraduate  before  you  began 


ocs? 

Mark  all  that  apply. 

o 

Agriculture 

o 

Architecture 

o 

Business 

o 

Communications 

o 

Computer  and  Infonnation  Science 

o 

Education 

o 

Engineering 
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o  English  Literature 
o  Foreign  language 
o  Health  Sciences 
o  Mathematics 
o  Military  Science 
o  Natural  Sciences 
o  Philosophy 
o  Physical  Sciences 
o  Pre-professional 
o  Psychology 
o  Public  Administration 
o  Recreation  and  Fitness  Studies 
o  Security  and  Protective  Services 
o  Social  Sciences 
o  Theology/Religion 

o  Other:  please  specify _ 

o  DOES  NOT  APPLY :  I  did  not  complete  college  prior  to  OCS 

7.  What  was  your  undergraduate  GPA  on  a  4-point  scale? 

o  Unsure  /  My  degree  program  did  not  use  a  4-point  GPA  scale 
o  Less  than  2.6 
o  2. 6-3.0 
o  3. 1-3.5 
o  3.6  or  above 

8.  When  you  were  in  OCS  were  you  ever  restarted  (i.e.,  repeated/recycled  a  portion  of  the  course 

or  been  assigned  to  a  new  class)? 

o  Yes 
o  No 

9.  Please  indicate  the  reason(s)  you  restarted/recycled  (choose  all  that  apply): 

o  Academic  failures 
o  Injury/medical  reasons 
o  Excessive  number  of  retests 
o  Leadership  deficiencies 
o  APFT  failure 

o  Administrative  reasons  (such  as  emergency  leave) 
o  Disciplinary  reasons 

o  Missed  excessive  instructional  periods  or  critical  instructional  blocks 
o  Other,  please  specify:  _ 

10.  When  you  were  in  BOLC-III  (now  called  BOLC  B)  were  you  ever  restarted/recycled  (i.e., 

repeated  a  portion  of  a  course  or  been  assigned  to  a  new  class)? 

o  Yes 
o  No 
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1 1 .  Please  indicate  the  reason(s)  you  restarted/recycled  (choose  all  that  apply): 
o  Academic  failures 

o  Injury/medical  reasons 
o  Excessive  number  of  retests 
o  Leadership  deficiencies 
o  APFT  failure 

o  Administrative  reasons  (such  as  emergency  leave) 
o  Disciplinary  reasons 

o  Missed  excessive  instructional  periods  or  critical  instructional  blocks 
o  Other,  please  specify:  _ 

12.  Did  you  graduate  from  BOLC-B/BOLC  III? 
o  Yes 

o  No 

13.  What  was  your  overall  percentage  for  graduation  from  BOLC-B/BOLC  III? 
o  91%  or  higher  (i.e.,  you  scored  in  the  top  10%  of  your  class) 

o  81-90% 
o  71-80% 
o  61-70% 
o  51-60% 
o  41-50% 
o  31-40% 
o  21-30% 
o  11-20% 

o  10%  or  less  (i.e.,  you  scored  in  the  bottom  10%  of  your  class) 
o  I  don’t  know 

14.  Did  you  make  the  Commandant's  List  (top  20%  of  your  class)  in  BOLC-B/  BOLC  III? 
o  Yes 

o  No 

15.  What  is  your  last  weapon's  qualification  score? 
o  Marksman  (MKM) 

o  Sharpshooter  (SPS) 
o  Expert  (EXP) 
o  Did  not  qualify 
o  Not  attempted 
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Branch  and  Assignments 


16.  What  branch  were  you  commissioned  in? 
o  Infantry 

o  Field  Artillery 
o  Signal  Corps 
o  Military  Police  Corps 
o  Military  Intelligence  Corps 
o  Adjutant  General  Corps 
o  Financial  Management 
o  Chemical  Corps 
o  Transportation 
o  Armor 
o  Ordnance  Corps 
o  Corps  of  Engineers 
o  Quartermaster  Corps 
o  Medical  Service  Corps 
o  Aviation 

o  Air  Defense  Artillery 

17.  When  was  your  branch  assigned? 
o  Before  entry  into  OCS 

o  At  the  beginning  of  OCS 
o  In  the  middle  of  OCS  (at  the  6-week  point) 
o  Near  the  end  of  OCS 

18.  How  was  your  branch  determined? 
o  HRC  assigned  my  branch  to  me. 

o  Given  my  OML  rank  at  OCS,  I  chose  my  branch  from  those  available, 
o  Other,  please  specify:  _ 

19.  Was  your  commissioning  branch  one  of  your  top  three  choices? 
o  Yes 

o  No 

20.  How  satisfied  are  you  with  your  branch  assignment? 
o  Very  satisfied 

o  Somewhat  satisfied 
o  Neither  satisfied  nor  dissatisfied 
o  Somewhat  dissatisfied 
o  Very  dissatisfied 
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21.  Since  commissioning,  what  jobs/assignments  have  you  held  for  at  least  90  days  or  are 
currently  assigned  to?  If  you  started  the  job  at  one  rank  and  continued  in  the  same  job  at  a  higher 
rank,  indicate  the  job  under  both  columns.  Choose  all  that  apply  for  each  rank. 


Job/Assignment 

2nd 

Lieutenant 

1st 

Lieutenant 

Captain 

Platoon/Team  Leader 

0 

0 

0 

Fire  Support/Fire  Direction  Officer 

0 

0 

0 

Pilot  in  Command 

0 

0 

0 

Specialty  Platoon  Leader 

0 

0 

0 

Company/Troop/Battery/Detachment  XO 

0 

0 

0 

Detachment  Commander 

0 

0 

0 

Company/Troop/Battery  Commander 

0 

0 

0 

Battalion/Squadron/Group  HQ  Level  Assistant/Special  Staff 
Officer/C  ommander 

0 

0 

0 

Brigade/BCT/Regiment  HQ  Level  Staff 
Assignment/Commander 

0 

0 

0 

Division/Installation  HQ  Level  Staff  Assignment 

0 

0 

0 

Corps/Theater  HQ  Level  Staff  Assignment 

0 

0 

0 

Service  School  Staff  Assignment 

0 

0 

0 

Small  Group  Leader  (SGL)/Tactical  Officer  (TAC) 

0 

0 

0 

Aide  de  Camp 

0 

0 

0 

Operations  Officer 

0 

0 

0 

Other  Assignment  (not  listed);  please  specify  below: 

0 

0 

0 

Deployment  History 

22.  Have  you  been  deployed  since  becoming  a  commissioned  officer? 
o  Yes 

o  No 

23.  How  many  total  months  have  you  been  deployed  since  becoming  a  commissioned  officer? 
o  Scale  Ranging  from  0  through  36  months 

24.  Being  deployed  has  increased  the  likelihood  that  I  will  remain  in  the  Army  beyond  my 
Active  Duty  Service  Obligation. 

o  Strongly  Agree 
o  Somewhat  Agree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Disagree 
o  Strongly  Disagree 
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Performance 


25.  Have  you  received  any  of  the  following  awards  and/or  decorations  since  becoming  a 
commissioned  officer?  (Choose  all  that  apply) 
o  Air  Medal 

o  Anny  Achievement  Medal 
o  Anny  Commendation  Medal 
o  Bronze  Star  Medal 
o  Combat  Action  Badge 
o  Combat  Field  Medical  Badge 
o  Combat  Infantryman  Badge 
o  Distinguished  Flying  Cross 
o  Expert  Field  Medical  Badge  Medal 
o  Good  Conduct  Medal 
o  Joint  Service  Achievement  Medal 
o  Joint  Service  Commendation  Medal 
o  Meritorious  Service  Medal 
o  Purple  Heart 
o  Silver  Star 
o  Soldier’s  Medal 

o  Have  not  received  an  award  or  decoration 

Please  indicate  whether  you  have  ever  received  the  following  since  becoming  a  commissioned 
officer: 


Yes 

No 

26.  a  General  Officer  Letter  or  Memorandum  of  Reprimand 
(GOLOR/GOMR)? 

0 

0 

27.  a  letter  of  concern? 

0 

0 

28.  formal  counseling  about  lack  of  effort? 

0 

0 

29.  formal  counseling  about  your  behavior  or  discipline? 

0 

0 

30.  formal  counseling  about  unsatisfactory  performance? 

0 

0 

3 1 .  a  referred  OER? 

0 

0 

32.  Has  a  senior  officer  or  rater  ever  recommended  you  for  command  on  the  OER? 
o  Yes 

o  No 

33.  In  your  most  recent  OER,  how  many  "NO"  ratings  in  Part  IV  -  Performance  Evaluation  - 
Professionalism  section  did  you  receive? 

o  0 
o  1 

o  2 
o  3 

o  4  or  more 
o  Cannot  remember 
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Did  you  receive  the  following  on  your  most  recent  OER? 


Yes 

No 

34.  "Outstanding  Performance  Must  Promote"  evaluation  from  a  rater? 

O 

0 

35.  "Unsatisfactory  Performance  Do  Not  Promote"  evaluation  from  a  rater? 

O 

0 

36.  "Best  Qualified"  evaluation  from  a  Senior  Rater? 

0 

0 

37.  "Do  Not  Promote"  evaluation  from  a  Senior  Rater 

0 

0 

38.  What  was  your  latest  Anny  Physical  Fitness  Test  (APFT)  score?  If  you  cannot  remember 
your  exact  score,  please  provide  your  best  estimate.  (Score  ranges  from  0-300) 

Score: 


In-Unit  Performance 


Please  rate  your  performance  in  each  of  the  following  areas  relative  to  your  peers  of  the  same 
rank  whom  you  know  or  work  with.  These  self-ratings  will  require  that  you  provide  estimates 
that  may  be  difficult  to  make  because  they  require  you  to  evaluate  not  only  yourself,  but  also 
others  in  your  peer  group.  Please  be  candid  and  give  us  your  honest  evaluation. 

39.  Physical  Fitness,  Health,  and  Endurance:  Effectively  engages  in  the  Anny  physical 
readiness  training  program  both  as  a  leader  and  a  participant.  Meets  height  and  weight  standards 
and  passes  the  Army  Physical  Fitness  Test.  Successfully  completes  all  physical  and  endurance 
requirements  of  platoon  commander  or  equivalent  position. 

o  Well  Below  Average  Bottom  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Average  Middle  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Well  Above  Average  Top  20%  of  Peer  Group 
o  Not  Applicable/Cannot  Rate 

40.  Army  Values,  Warrior  Ethos,  and  Ethics:  Embodies,  lives,  and  defends  Anny  Values  and 
Warrior  Ethos.  Adheres  to  Anny  officer  standards  of  conduct  and  holds  to  a  highly  ethical  code 
of  behavior. 

o  Well  Below  Average  Bottom  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Average  Middle  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Well  Above  Average  Top  20%  of  Peer  Group 
o  Not  Applicable/Cannot  Rate 
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41.  Technical  Competence:  Effectively  demonstrates  technical  skills  for  performing  branch- 
specific  requirements. 

o  Well  Below  Average  Bottom  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Average  Middle  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Well  Above  Average  Top  20%  of  Peer  Group 
o  Not  Applicable/Cannot  Rate 

42.  Tactical  Competence:  Effectively  employs  Warrior  Tasks  and  Battle  Drills,  tactical  skills, 
and  makes  necessary  contributions  to  solving  tactical  problems. 

o  Well  Below  Average  Bottom  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Average  Middle  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Well  Above  Average  Top  20%  of  Peer  Group 
o  Not  Applicable/Cannot  Rate 

43.  Leadership:  Effectively  performs  as  an  Anny  leader  in  all  positions  and  assignments. 
Applies  Troop  Leading  Procedures  (TLP)  in  execution  of  leadership  roles.  Supports  subordinate, 
peer  and  superior  leaders. 

o  Well  Below  Average  Bottom  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Average  Middle  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Well  Above  Average  Top  20%  of  Peer  Group 
o  Not  Applicable/Cannot  Rate 

44.  Officership:  Inculcates  the  characteristics  of  being  an  officer  to  include  customs,  traditions 
and  heritage.  Develops  knowledge  and  awareness  of  the  Army  as  an  institution  to  include 
military  operation  theory  and  management,  personnel  and  career  management,  military  justice 
and  discipline. 

o  Well  Below  Average  Bottom  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Average  Middle  20%  of  Peer  Group 
o  Next  20%  of  Peer  Group 
o  Well  Above  Average  Top  20%  of  Peer  Group 
o  Not  Applicable/Cannot  Rate 
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Career  Intent  and  Attitudes 


Please  choose  the  response  that  indicates  the  extent  to  which  you  agree  with  each  of  the 

following  statements  regarding  your  feelings  about  the  Anny. 

45.  How  would  you  rate  your  current  level  of  morale? 
o  Very  high 
o  High 
o  Moderate 
o  Low 
o  Very  Low 

46. 1  feel  'like  part  of  the  family'  in  the  Anny. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

47. 1  feel  'emotionally  attached’  to  the  Anny. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

48.  The  Anny  has  a  great  deal  of  personal  meaning  for  me. 
o  Strongly  Disagree 
o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

49. 1  feel  a  strong  sense  of  belonging  to  the  Anny. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

50. 1  am  afraid  of  what  might  happen  if  I  quit  the  Anny  without  having  another  job  lined  up. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 
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5 1 .  Too  much  of  my  life  would  be  disrupted  if  I  decided  I  wanted  to  leave  the  Anny  now. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

52.  It  would  be  too  costly  for  me  to  leave  the  Army  in  the  near  future, 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

53.  One  of  the  problems  of  leaving  the  Anny  would  be  the  lack  of  available  alternatives, 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

54.  In  general,  the  Army  has  kept  its  promises  to  me. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

55.  In  most  instances,  the  Anny  has  not  met  its  obligations  to  me. 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

56.  In  general,  I  have  kept  my  promises  to  the  Army, 
o  Strongly  Disagree 

o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 


B-l  1 


57.  In  most  instances,  I  have  not  met  my  obligations  to  the  Army. 

o  Strongly  Disagree 
o  Somewhat  Disagree 
o  Neither  Agree  nor  Disagree 
o  Somewhat  Agree 
o  Strongly  Agree 

58.  What  is  the  highest  rank  you  think  you  will  achieve  in  your  Army  career? 
o  Captain 

o  Major 

o  Lieutenant  Colonel 
o  Colonel 
o  General  Officer 

59.  Which  of  the  following  best  describes  your  current  active  duty  career  intentions? 
o  I  plan  to  stay  in  the  Anny  beyond  20  years 

o  I  plan  to  stay  in  the  Anny  until  retirement  (e.g.,  20  years  or  when  eligible  to  retire) 
o  I  plan  to  stay  in  the  Anny  beyond  my  obligation,  but  I  am  undecided  about  staying  until 
retirement 

o  I  am  undecided  whether  I  will  stay  in  the  Army  upon  completion  of  my  obligation 
o  I  will  probably  leave  the  Army  upon  completion  of  my  obligation 
o  I  will  definitely  leave  the  Anny  upon  completion  of  my  obligation 

60.  Have  you  submitted  a  request  release  from  active  duty  (REFRAD)? 

o  I  do  not  plan  to  submit  a  request  REFRAD  in  the  near  future,  because  I  expect  to 
continue  my  Anny  service  for  more  than  6  months  beyond  my  ADSO. 
o  Not  yet,  but  I  plan  to  submit  one  in  time  to  separate  within  6  months  after  my  ADSO  ends, 
o  Yes,  I  have  submitted  a  request  REFRAD  and  plan  to  separate  within  6  months  after  my 
ADSO  ends. 

61.  If  you  plan  to  leave  active  duty  service,  please  indicate  the  reason(s)  you  plan  to  leave  the 
Army  after  your  obligation  is  completed  (mark  all  that  apply). 

o  Going  into  the  Anny  Reserve 
o  Going  into  the  Anny  National  Guard 
o  Have  achieved  what  I  wanted  in  an  Anny  career 
o  Too  many  deployments 
o  Offered  a  good  civilian  job 
o  Quality  of  military  leadership 
o  Lack  of  stability/predictability  of  Army  life 
o  Away  from  family  too  much 
o  Spouse  wants  me  to  leave  the  Army 
o  Lack  of  promotion  potential 
o  Could  not  get  the  branch  I  wanted 
o  Could  not  get  the  assignment  I  wanted 
o  Could  not  get  graduate  education 
o  Lack  of  employment  opportunities  for  my  spouse 
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o  Other,  please  specify: 
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APPENDIX  C 


SUBSEQUENT  DATA  COLLECTIONS  AND  INSTRUMENTS 
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Supervisor  Ratings  of  In-Unit  Performance 


As  with  ARI’s  other  officer-selection  projects,  obtaining  quality  in-unit  performance  data 
for  use  in  test  validation  has  been  an  ongoing  challenge.  Perfonnance-related  infonnation 
available  from  automated  personnel  records  is  generally  limited  and  deficient,  as  such  measures 
are  intended  for  administrative  purposes  and  not  personnel  research.  Some  useful  performance- 
related  data  can  be  obtained  directly  through  self-reports  using  surveys  (as  with  the  In-Unit 
Survey  described  in  Chapter  2),  but  this  data  collection  approach  also  has  significant  limitations. 

Collecting  Supervisor  Ratings  Data 

Accordingly,  we  sought  a  means  for  collecting  supervisor  ratings  to  enrich  the  criterion 
space  for  our  OCS  longitudinal  validation  sample.  The  approach  needed  to  be  technically  sound, 
cost  effective,  and  practical.  This  goal  was  made  more  challenging  by  the  fact  that  the  officers  in 
the  2008  OCS  cohort  were  spread  out  in  locations  throughout  the  world.  In  addition,  ARI’s 
Internal  Review  Board  (IRB)  guidance  for  this  research  required  that  we  obtain  explicit 
permission  from  potential  ratees  prior  to  requesting  any  performance  information  from  their 
supervisors.  This  requirement  further  complicated  the  logistical  challenges,  and  we  recognized 
that  collecting  supervisor  ratings  would  likely  be  a  labor-intensive  and  potentially  expensive 
process. 

As  an  initial  step  in  this  effort,  we  developed  in-unit  performance  rating  materials  and 
administered  them  to  several  supervisors  of  the  OCS  cohort  members  (with  pennission  from  the 
ratees)  during  face-to-face  sessions  at  one  Anny  post.  We  sought  this  particular  post  because  a 
relatively  large  number  of  individuals  in  our  target  sample  of  OCS-commissioned  officers  were 
stationed  there  at  the  time.  Our  experience  there  helped  infonn  both  the  refinement  of  the 
measures  and  the  refinement  of  our  data  collection  approach.  We  concluded  that  it  would  not  be 
feasible  to  pursue  on-site  data  collections,  due  to  their  logistical  challenges  and  time-consuming 
nature.  Relatively  few  ratings  were  collected  given  the  investment  in  time  and  resources.11 

Through  a  process  which  evolved  over  several  months,  we  later  developed  a  web-based, 
semi-automated  performance  rating  system,  which  is  being  fielded  at  the  time  of  this  writing.  In 
the  first  step  of  this  approach,  participants  from  the  OCS  2008  cohort  are  contacted  by  e-mail 
and  asked  pennission  to  have  ratings  collected.  If  they  agree  to  participate,  they  are  asked  to 
provide  their  supervisor’s  contact  information  on  a  secure  server.  Next,  the  supervisors  are 
contacted  by  e-mail  and  asked  to  complete  confidential  ratings  of  the  Junior  Officer  Perfonnance 
Rating  Scales  (JOPRS)  through  a  secure  website.  If  this  performance  rating  system  approach 
proves  to  be  feasible  for  collecting  data  from  small  samples  of  ratees,  we  anticipate  that  it  will  be 
used  not  only  to  obtain  rating  data  for  our  OCS  cohort  members,  but  eventually  for  other  ARI 
officer  selection  research  efforts  also. 


1 1  The  on-site  data  collection  was  made  more  difficult  by  the  relatively  high  frequency  of  deployments  among  the 
sample  of  officers  we  were  seeking.  This  was  largely  due  to  the  Army’s  operational  requirements  at  that  time. 
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Junior  Officer  Performance  Rating  Scales 

The  Junior  Officer  Performance  Rating  Scales  (JOPRS)  were  extracted  from  a  larger  set 
of  rating  scales  developed  for  the  Anny  Officer  Predictor  and  Criterion  project  (Paullin  et  ah, 
2012).  The  JOPRS  contains  rating  scales  for  7  foundational  skills,  24  leadership  duties,  and  18 
management  duties.  The  foundational  skills  are  aimed  at  major  dimensions  of  performance  from 
Campbell,  McCloy,  Oppler,  and  Sager  (1993)  model  of  performance,  which  was  largely  based 
on  the  results  of  the  Army’s  Project  A  (Campbell  &  Knapp,  2001).  The  leadership  and 
managerial  duties  were  originally  developed  in  a  job  analysis  of  officer  jobs  (Paullin  et  ah,  2010) 
and  were  organized  into  breakdowns  of  leadership  and  managerial  dimensions  in  Campbell’s 
recent  expansion  of  his  perfonnance  model  (Campbell,  2012).  For  the  skill  and  duty  items, 
respondents  rate  the  junior  officer  using  a  5-point  rating  scale,  ranging  from  “Well  Below 
Average  (Bottom  20%  of  Peer  Group)”  to  “Well  Above  Average  (Top  20%  of  Peer  Group).” 

A  final  JOPRS  item  asks  respondents  to  rate  the  officer’  likely  effectiveness  in  a  field 
grade  leadership  position  using  an  8-point  rating  scale  ranging  from  “Likely  to  be  a  poor  to 
marginal  performer”  to  “Likely  to  be  a  truly  exceptional  performer.”  Readers  interested  in  more 
detailed  information  on  the  development  of  the  JOPRS  should  consult  the  Officer  Predictor  and 
Criterion  project  technical  report  (Paullin  et  ah,  2012).  The  solicitation  emails  and  full  JOPRS 
data  collection  materials  are  presented  on  subsequent  pages. 
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Request  for  JOPRS  Participation  (Email  Sent  to  Ratees) 


SUBJECT:  Performance  Ratings  for  OCS  Selection  Research 
Dear  OCS  Commissioned  Officer: 

The  U.S.  Army  Research  Institute  (ARI),  in  partnership  with  OCS,  is  continuing  the  evaluation 
of  assessments  to  enhance  the  selection  of  the  most  promising  candidates  for  OCS.  Beginning  in 
2008,  you  participated  in  a  related  ARI  research  effort.  We  are  infonning  you  of  additional 
efforts  to  collect  information  related  to  OCS  and  perfonnance  as  an  Army  officer. 

With  your  permission,  we  would  like  to  obtain  "off  the  record"  supervisor  ratings  of  your  in-unit 
perfonnance.  These  confidential  ratings  would  be  used  to  further  evaluate  the  assessment 
measures  you  completed  at  the  beginning  of  OCS.  Your  participation  in  this  effort  is  voluntary. 
If  collected,  the  ratings  would  be  used  for  research  purposes  only.  They  would  not  be  used  in  any 
way  to  evaluate  you  or  any  other  individual.  Your  supervisor's  ratings  would  not  become  part  of 
your  Anny  record  and  would  have  no  impact  on  your  Army  career.  Beyond  ARI,  no  Anny 
agency  would  ever  have  any  control  or  access  to  the  rating  data  being  collected  on  individual 
officers. 

This  is  a  unique  opportunity  for  us  to  capture  officer  performance  information  that  will  be  very 
beneficial  to  the  Army  as  we  move  our  experimental  OCS  selection  tests  toward  implementation. 

Thank  you  for  your  consideration;  your  willingness  to  participate  would  be  greatly  appreciated. 

At  your  earliest  convenience,  please  click  on  the  link  below  and  indicate  whether  you  approve  or 
decline  participation.  Before  accessing  the  website,  you  will  need  to  enter  the  user  ID  code 
shown  below. 

If  you  have  any  questions  about  this  request,  please  feel  free  to  contact  us  directly  using  the 
phone  numbers  provided  below. 

Very  Respectfully, 

Signature  and  contact  information  included 
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Request  for  JOPRS  Participation  (Ratee  Survey  Content) 


Are  you  [participant  name]? 

□  Yes 

□  No 

<if  answer  yes> 

With  your  permission,  we  would  like  to  obtain  “off  the  record”  supervisor  ratings  of  your  in-unit 
perfonnance.  These  ratings  would  be  used  to  further  evaluate  the  OCS  assessment  measures,  and 
kept  confidential  and  private.  They  never  become  part  of  your  Anny  record. 

Please  indicate  below  whether  or  not  you  approve  of  confidential  perfonnance  ratings  being 
collected  on  you. 

□  Yes,  1  approve 

□  No,  1  do  not  approve 
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<for  those  who  are  willing  to  participate> 


Please  provide  the  names  and  contact  information  for  a  supervisor  who  can  provide  ratings  of 
your  in-unit  perfonnance.  Raters  should: 

•  be  your  superior  in  rank  and 

•  have  closely  observed  your  performance  over  time 

Often  this  is  a  current  supervisor;  however,  a  former  supervisor  would  be  preferred  if  he/she  has 
had  better  opportunity  to  observe  your  perfonnance.  If  more  than  one  individual  could  serve  as  a 
rater  for  you,  please  provide  two  names. 

Supervisor  1: 

Rank:  [drop  down  menu:  02,  03,  04,  05,  06,  Mr.,  Ms.,  Mrs.,  I  don’t  know] 

Last  Name: _ 

First  Name: _ 

Middle  Initial: _ 

AKO  email  address: _ @us. anny.mil 

Enterprise  email  address: _ [drop  down  menu:  .mil@mail.mil,  .civ@mail.mil] 

Current  location: _ 

Supervisor  2: 

Rank:  [drop  down  menu:  02,  03,  04,  05,  06,  Mr.,  Ms.,  Mrs.,  I  don’t  know] 

Last  Name: _ 

First  Name: _ 

Middle  Initial: _ 

AKO  email  address: _ @us. anny.mil 

Enterprise  email  address: _ [drop  down  menu:  .mil@mail.mil,  .civ@mail.mil] 

Current  location: _ 

<closing  screen> 

Thank  you.  Your  willingness  to  participate  is  greatly  appreciated 
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Request  for  JOPRS  Participation  (Email  Sent  to  Raters) 


[Rank]  [Last  name]: 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  is  evaluating  a 
new  selection  battery  for  OCS.  In  this  research  phase,  we  are  linking  test  scores  collected  from 
OCS  candidates  in  2008  with  their  in-unit  performance  as  junior  officers. 

Our  research  team  needs  your  help  in  providing  confidential  "off  the  record"  perfonnance  ratings 
of  [Rank]  [Full  Name].  [Rank]  [Last  name]  has  given  approval  for  us  to  contact  you.  The  task 
should  take  less  than  ten  minutes  of  your  time. 

Your  participation  is  voluntary  and  the  data  would  be  kept  confidential  for  research  purposes 
only. 

This  is  a  unique  opportunity  for  us  to  capture  officer  performance  information  that  will  be  very 
beneficial  to  the  Anny  as  we  move  our  experimental  OCS  selection  tests  toward  implementation. 

Thank  you  for  your  consideration;  your  willingness  to  provide  the  performance  ratings  would  be 
greatly  appreciated. 

At  your  earliest  convenience,  please  click  on  the  link  below  and  indicate  whether  you  would  be 
able  to  provide  us  with  the  requested  ratings.  You  will  need  to  enter  the  USER  ID  code  shown 
below  when  you  indicate  your  intention  to  participate  (or  not  participate). 

If  you  have  any  questions  about  this  request,  please  feel  free  to  contact  us  directly. 

Very  Respectfully, 

Signature  and  contact  information  included 


crating  materials  are  presented  on  subsequent  pages> 
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Privacy  Act  Statement 


AUTHORITY :  The  Department  of  the  Anny  may  collect  the  information  requested  in  this  study 
under  the  authority  of  10  United  States  Code,  Section  2358,  "Research  and  Development 
Projects."  In  accordance  with  the  Privacy  Act  of  1974  (Public  Law  93-579),  this  notice  informs 
you  of  the  purpose,  use,  and  confidentiality  of  this  session. 

PURPOSE:  The  purpose  of  this  research  is  to  develop  and  evaluate  measures  for  selecting 
officer  candidates.  You  are  being  asked  to  provide  perfonnance  ratings  of  an  officer  who  is  or 
was  under  your  supervision.  Your  ratings  will  be  used  to  validate  the  selection  measures  that  this 
individual  completed  upon  entry  into  OCS  in  2008. 

WHAT  IS  BEING  ASKED  OF  YOU:  You  are  being  asked  to  complete  a  questionnaire 
wherein  you  will  rate  the  performance  of  an  individual  who  is  or  was  under  your  supervision. 

ROUTINE  USES:  Findings  from  this  questionnaire  may  be  used  to  fonnulate,  refine,  and 
improve  future  officer  candidate  selection  tests.  The  purpose  of  the  data  is  to  develop  and 
evaluate  new  selection  measures,  and  the  research  findings  may  be  provided  to  Army  leaders  to 
develop  policy  initiatives  regarding  officer  selection. 

DISCLOSURE:  The  data  collected  will  be  used  for  research  purposes  only.  They  will  not  be 
used  to  evaluate  you  or  any  other  individual.  Only  aggregated  data  will  be  reported.  Your 
responses  will  not  become  part  of  anyone’s  Anny  record  and  will  have  no  impact  on  anyone’s 
Anny  career.  Participating  in  this  rating  task  is  voluntary  and  you  may  choose  at  any  time  not  to 
participate.  There  is  no  penalty  for  choosing  not  to  participate. 

CONFIDENTIALITY:  We  will  not  identify  you,  or  attribute  your  responses  to  you.  We  will 
NOT  include  your  name  or  other  personally  identifiable  information  in  our  reports.  ARI  will 
safeguard  your  privacy  by  dropping  any  personal  identifiers  that  could  be  traced  back  to  you. 


CONTACT:  For  further  infonnation  about  this  project  or  rights  as  a  participant,  send  email  to: 
£P3SSHflSGGM9Mfl’  Reference  project  name:  Measures  for  Selecting  Officer  Candidates 
-  Junior  Officer  Perfonnance  Ratings. 
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Informed  Consent 


PURPOSE  OF  THE  RESEARCH  STUDY:  The  purpose  of  this  research  is  to  develop  and 
evaluate  measures  for  selecting  officer  candidates.  You  are  being  asked  to  provide  perfonnance 
ratings  of  an  officer  who  is  or  was  under  your  supervision.  Your  ratings  will  be  used  to  validate 
the  selection  measures  that  this  individual  completed  upon  entry  into  OCS  in  2008. 

WHAT  YOU  WILL  BE  ASKED  TO  DO  IN  THIS  RESEARCH:  In  this  research,  you  are 
being  asked  to  complete  a  questionnaire  wherein  you  will  rate  the  performance  of  an  individual 
who  is  or  was  under  your  supervision.  Your  responses  will  be  kept  strictly  confidential,  and  will 
be  used  for  research  purposes  only.  No  infonnation  regarding  any  individual’s  responses  on  this 
survey  will  be  provided  to  anyone’s  chain  of  command. 

VOLUNTARY  PARTICIPATION:  Your  participation  is  voluntary;  if  you  do  not  want  to 
participate  there  will  be  no  penalty  or  loss  of  benefits  to  which  you  are  otherwise  entitled.  You 
have  the  right  to  withdraw  from  the  research  at  any  time  without  bias  or  penalty  of  any  kind. 

TIME  REQUIRED:  Up  to  10  minutes 

RISKS:  All  questionnaire  items  ask  about  your  judgments  and  views  rather  than  about 
infonnation  that  would  be  considered  personal  in  nature.  We  do  not  anticipate  that  your 
participation  has  any  risks  greater  than  those  encountered  in  everyday  activities. 

BENEFITS:  Your  responses  will  help  in  developing  measures  that  can  be  used  to  select 
candidates  who  are  likely  to  perform  well  as  junior  officers,  fit  well  in  the  Army’s  culture, 
demonstrate  leadership  potential  for  higher  ranks,  and  pursue  a  long-tenn  Army  career. 

COMPENSATION:  No  compensation  is  provided  for  your  participation. 

WHOM  TO  CONTACT  IF  YOU  HAVE  QUESTIONS  ABOUT  THE  QUESTIONNAIRE: 

You  should  send  your  questions  to  Reference  project  name: 

Measures  for  Selecting  Officer  Candidates  -  Junior  Officer  Perfonnance  Ratings. 

WHOM  TO  CONTACT  ABOUT  YOUR  RIGHTS  IN  THIS  STUDY:  You  should  send  your 
questions  to  Reference  project  name:  Measures  for  Selecting 

Officer  Candidates  -  Junior  Officer  Performance  Ratings. 

INFORMED  CONSENT:  If  you  agree  to  provide  a  perfonnance  rating,  please  click  on  the  “I 
agree  to  participate”  button  below  to  access  the  rating  fonn. 


I  agree  to  participate 


I  do  not  agree  to  participate 
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Junior  Officer  Performance  Rating  Scales 


You  will  be  rating  the  following  Officer(s). 

[name  1] 

[name  2] 

Background  Questions 

Instructions:  Please  complete  this  fonn  by  selecting  the  appropriate  response  option. 

Please  rate  the  extent  to  which  you  have  had  opportunity  to  observe  [officer  name]  you 
will  be  rating. 

□  Not  enough  to  judge  any  aspects  of  the  officer’s  performance 

□  Enough  to  judge  some  aspects  of  the  officer’s  performance 

□  Enough  to  judge  most  aspects  of  the  officer’s  performance 

□  Enough  to  judge  almost  all  aspects  of  the  officer’s  performance 

Which  of  the  following  best  describes  your  relationship  with  [officer  name]? 

□  1  am  currently  his/her  supervisor. 

□  1  was  previously  his/her  supervisor. 

□  1  never  supervised  him/her  but  have  observed  his/her  in-unit  performance. 

□  Other;  Please  specify: _ 

Please  indicate  your  rank. 

□  03 

□  04 

□  05 

□  06 

Please  indicate  your  time  of  service  in  the  Army. 


Years 

0 

0 

1 

1 

2 

2 

3 

3 

4 

5 

6 

7 

8 

9 

Months 

0 

0 

1 

1 

2 

3 

4 

5 

6 

7 

8 

9 
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Please  indicate  your  Branch  below. 


□  Adjutant  General  Corps 

□ 

□  Air  Defense  Artillery 

□ 

□  Armor 

□ 

□  Aviation 

□ 

□  Chemical  Corps 

□ 

□  Coips  of  Engineers 

□ 

□  Field  Artillery 

□ 

□  Financial  Management 

□ 

□  Other,  Please  specify: 

Infantry 

Medical  Services  Coips 
Military  Intelligence  Corps 
Military  Police  Coips 
Ordnance  Corps 
Quartermaster  Coips 
Signal  Corps 
Transportation 
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Consider  the  officer’s  typical  performance  over  the  time  you  have  observed  him/her.  Indicate  how  well  the  officer  performs  relative 
to  other  officers  with  similar  post-commissioning  experience  that  you  have  observed  using  the  rating  scale  shown  below.  For 
each  of  the  49  tasks,  select  the  rating  that  best  reflects  the  officer’s  performance.  If  you  have  not  observed  the  officer’s  performance 
sufficiently  to  make  a  rating  on  a  given  task,  select  “Not  Observed/Cannot  Rate.” 


Well 

Well 

Below 

Below 

Average 

Above 

Above 

Not 

Average 

Average 

Observed/ 

How  well  does  this  officer  perforin  relative  to  other  officers  with 

Bottom 

Next  lower 

Middle 

Next  higher 

Top 

Cannot 

similar  post-commissioning  experience? 

20%  of 

20%  of 

20%  of 

20%  of 

20%  of 

Rate 

Peer  Group 

Peer  Group 

Peer  Group 

Peer  Group 

Peer  Group 

FOUNDATIONAL  DUTIES 

1. 

Performs  technical  and  tactical  duties  proficiently 

□ 

□ 

□ 

□ 

□ 

□ 

2. 

Performs  core  warrior  tasks  proficiently 

□ 

□ 

□ 

□ 

□ 

□ 

3. 

Communicates  clearly  in  writing 

□ 

□ 

□ 

□ 

□ 

□ 

4. 

Communicates  clearly  when  speaking 

□ 

□ 

□ 

□ 

□ 

□ 

5. 

Demonstrates  effort  and  willingness  to  keep  working  under  adverse 
conditions 

□ 

□ 

□ 

□ 

□ 

□ 

6. 

Demonstrates  self-control  and  personal  discipline 

□ 

□ 

□ 

□ 

□ 

□ 

7. 

Maintains  physical  fitness,  strength,  and  weight  standards 

□ 

□ 

□ 

□ 

□ 

□ 

LEADERSHIP:  CONSIDERATION,  SUPPORT,  PERSON- 
CENTERED 

8. 

Balances  the  requirements  of  the  mission  with  the  welfare  of  others 

□ 

□ 

□ 

□ 

□ 

□ 

9. 

Builds  trust  and  working  relationships  with  other  units  within  the 

Army 

□ 

□ 

□ 

□ 

□ 

□ 

10. 

Resolves  conflict  through  consensus-building  and  negotiation. 

□ 

□ 

□ 

□ 

□ 

□ 

11. 

Listens  actively  to  others 

□ 

□ 

□ 

□ 

□ 

□ 

12. 

Fosters  effective  teamwork  and  cooperation. 

□ 

□ 

□ 

□ 

□ 

□ 

13. 

Creates  a  fair  and  inclusive  work  environment  that  acknowledges  and 
makes  effective  use  of  diverse  backgrounds  and  skills 

□ 

□ 

□ 

□ 

□ 

□ 

14. 

Demonstrates  a  concern  for  people  and  their  well-being 

□ 

□ 

□ 

□ 

□ 

□ 

15. 

Accepts  and  effectively  deals  with  reasonable  setbacks  and  failures  of 
subordinates  and  others 

□ 

□ 

□ 

□ 

□ 

□ 

16. 

Builds  trust  and  working  relationships  with  individuals,  groups,  and 
organizations  outside  the  Army 

□ 

□ 

□ 

□ 

□ 

□ 
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How  well  does  this  officer  perform  relative  to  other  officers  with 
similar  post-commissioning  experience? 


LEADERSHIP:  INITIATING  STRUCTURE,  GUIDING, 
DIRECTING 

17.  Directs  others  to  work  toward  mission  accomplishment 

18.  Keeps  subordinates,  superiors  and  others  informed  of  relevant 
information 

LEADERSHIP:  GOAL  EMPHASIS 

19.  Communicates  mission  goals  and  objectives 

20.  Influences  subordinates  to  work  toward  mission  accomplishment 

21.  Influences  superiors  to  adopt  a  position  or  course  of  action 

LEADERSHIP:  EMPOWERMENT,  FACILITATION 

22.  Empowers  subordinates  to  exercise  initiative  and  take  ownership  over 
work 

23.  Creates  an  environment  that  fosters  innovative  and  critical  thinking 

24.  Encourages  open  communications  and  diverse  points  of  view 

LEADERSHIP:  TRAINING,  COACHING 

25.  Trains  subordinates 

26.  Counsels,  coaches,  and  mentors  subordinates  and  others 

LEADERSHIP:  SERVING  AS  A  MODEL 

27.  Leads  with  confidence  in  adverse  situations 

28.  Maintains  own  mental  health  and  well-being 

29.  Demonstrates  self-awareness  and  self-understanding 

30.  Engages  in  self-development  activities 

3 1 .  Demonstrates  cultural  sensitivity  in  communication 


Well 

Below 

Average 

Bottom 
20%  of 
Peer  Group 

Below 

Next  lower 
20%  of 
Peer  Group 

Average 

Middle 
20%  of 
Peer  Group 

Above 

Next  higher 
20%  of 
Peer  Group 

Well 

Above 

Average 

Top 

20%  of 
Peer  Group 

Not 

Observed/ 

Cannot 

Rate 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

C-14 


How  well  does  this  officer  perform  relative  to  other  officers  with 
similar  post-commissioning  experience? 


MANAGEMENT:  GOAL  SETTING,  PLANNING, 
ORGANIZING,  AND  BUDGETING 

32.  Establishes  mission  goals  and  objectives 

33.  Finds  ways  to  minimize  or  eliminate  distractions  or  low  priority  tasks 
that  get  in  the  way  of  mission  accomplishment 

34.  Plans/delegates  work  assignments  in  accordance  with  the  skills  and 
developmental  needs  of  others 

MANAGEMENT:  COORDINATION 

35.  Prioritizes,  organizes,  and  coordinates  tasks  for  oneself,  individuals, 
teams,  and  other  organizational  units 

MANAGEMENT:  MONITORING  UNIT  EFFECTIVENESS 

36.  Monitors  and  evaluates  operational  effectiveness 

37.  Evaluates  and  enforces  performance  of  subordinates 

MANAGEMENT:  EXTERNAL  REPRESENTATION 

38.  Develops  and  maintains  relevant  geopolitical  awareness 

39.  Develops  and  maintains  cultural  awareness 

MANAGEMENT:  STAFFING 

40.  Assesses  developmental  needs  of  subordinates  and  others  and 
facilitates  their  ongoing  development 

4 1 .  Identifies  and  leverages  opportunities  to  improve  individual,  group, 
and  organizational  performance 

MANAGEMENT:  DECISION  MAKING,  PROBLEM  SOLVING, 
AND  STRATEGIC  INNOVATION 

42.  Generates  innovative  solutions  to  problems 

43.  Uses  judgment  and  logic  to  solve  problems 


Well 

Below 

Average 

Bottom 
20%  of 
Peer  Group 

Below 

Next  lower 
20%  of 
Peer  Group 

Average 

Middle 
20%  of 
Peer  Group 

Above 

Next  higher 
20%  of 
Peer  Group 

Well 

Above 

Average 

Top 

20%  of 
Peer  Group 

Not 

Observed/ 

Cannot 

Rate 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

C-15 


How  well  does  this  officer  perforin  relative  to  other  officers  with 
similar  post-commissioning  experience? 

Well 

Below 

Average 

Bottom 
20%  of 
Peer  Group 

Below 

Next  lower 
20%  of 
Peer  Group 

Average 

Middle 
20%  of 
Peer  Group 

Above 

Next  higher 
20%  of 
Peer  Group 

Well 

Above 

Average 

Top 

20%  of 
Peer  Group 

Not 

Observed/ 

Cannot 

Rate 

44. 

Identifies  and  adjusts  to  external  influences  on  the  mission  or 
organization 

□ 

□ 

□ 

□ 

□ 

□ 

45. 

Makes  decisions  that  reflect  an  understanding  of  sphere  of  influence 

□ 

□ 

□ 

□ 

□ 

□ 

46. 

Creates  and  disseminate  a  vision  of  the  future 

□ 

□ 

□ 

□ 

□ 

□ 

47. 

Expands  knowledge  of  technical,  technological,  and  tactical  areas 

MANAGEMENT:  COMMITMENT  AND  COMPLIANCE 

□ 

□ 

□ 

□ 

□ 

□ 

48. 

Demonstrates  the  Army  Values  and  Warrior  Ethos 

MANAGEMENT:  ADMINISTRATION 

□ 

□ 

□ 

□ 

□ 

□ 

49. 

Identifies,  obtains,  allocates,  and  manages  resources 

□ 

□ 

□ 

□ 

□ 

□ 

50.  Please  rate  this  officer’s  likely  effectiveness  in  a  field  grade  leadership  position  (e.g.,  Battalion  Commander,  Brigade 
Commander). 


Likely  to  be  a  poor  to 

Likely  to  be  a  satisfactory  to 

Likely  to  be  a  very  good  to 

Likely  to  be  a  truly 

Cannot  Rate: 

marginal  performer 

good  performer 

excellent  performer 

exceptional  performer 

Inadequate 
opportunity 
to  observe 
this  person 

1 

2 

3 

4 

5 

6 

7 

8 

N/A 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Thank  you  for  making  your  ratings. 


APPENDIX  D 


BIVARIATE  CORRELATION  TABLES 


D-l 


Table  D.l.  Full  Information  Maximum  Likelihood  (FIML) 


Scale 

i 

2 

3 

4 

5 

6 

1 

RBI:  Peer  Leadership 

- 

.47 

.36 

.15 

-.15 

.47 

2 

RBI:  Achievement 

.49 

- 

.25 

.10 

-.26 

.41 

3 

RBI:  Fitness  Motivation 

.29 

.25 

- 

.26 

-.04 

.42 

4 

RBI:  Stress  Tolerance 

.07 

.11 

.17 

- 

-.31 

.33 

5 

RBI:  Hostility  to  Authority 

-.10 

-.26 

.12 

-.37 

- 

-.24 

6 

RBI:  Self-Efficacy 

.50 

.57 

.26 

.20 

-.18 

- 

7 

RBI:  Army  Affective  Commitment 

.31 

.45 

.13 

.15 

-.22 

.35 

8 

RBI:  Continuance  Commitment 

-.18 

-.06 

-.09 

-.10 

-.02 

-.19 

9 

RBI:  Traditional  Values 

.15 

.08 

.16 

-.12 

.11 

.17 

10 

RBI:  Validity  Scale 

.18 

.24 

.15 

.20 

-.24 

.26 

11 

Values  :  Rugged  Leadership 

.31 

.44 

.25 

-.08 

-.02 

.35 

12 

Values  :  Flexibility/Choice 

-.01 

.06 

-.05 

-.36 

.12 

.03 

13 

Values  :  Structure/Recognition 

.08 

.18 

.06 

-.20 

.09 

.12 

14 

Values  :  Altruism/Benevolence 

.27 

.49 

.21 

-.01 

-.10 

.39 

15 

Values  :  Skill  Development 

.32 

.50 

.03 

.02 

-.16 

.29 

16 

Values  :  Job  Security 

-.05 

.12 

-.06 

-.24 

-.03 

.07 

17 

Values  :  Teamwork 

.27 

.38 

.11 

-.07 

-.07 

.33 

18 

Situational  Judgment  Test 

.27 

.22 

.17 

.12 

-.13 

.17 

19 

Promotion  Rate  1  st  Lieutenant 

.01 

-.03 

.13 

.05 

.02 

.03 

20 

Std.  Total  Awards 

- 

- 

- 

- 

- 

- 

21 

Std.  Weighted  Awards 

- 

- 

- 

- 

- 

- 

22 

AFQT 

-.06 

-.16 

.08 

.13 

.01 

-.18 

23 

IOT  Graduation 

-.02 

.03 

-.07 

.06 

-.03 

.10 

24 

IOT  Recycles 

.08 

.02 

.06 

-.05 

.00 

-.06 

25 

Personal  Discipline 

.25 

.06 

.06 

-.01 

-.03 

.13 

26 

Leadership 

-.10 

.04 

.06 

-.04 

.01 

.02 

27 

Career  Continuance  Intentions 

-.01 

-.09 

-.03 

.03 

.11 

.09 

28 

Branch-Specific  Tech  Knowledge 

.23 

.05 

.10 

-.03 

-.08 

.19 

29 

Army-Wide  Tech  Knowledge 

.11 

.05 

.19 

.03 

-.05 

.24 

30 

Physical  Fitness 

.21 

.13 

.38 

.04 

-.04 

.11 

31 

Branch  Satisfaction 

.01 

.17 

-.02 

.22 

-.16 

.20 

32 

Army  Commitment 

.21 

.26 

.09 

.08 

-.06 

.14 

33 

Morale 

.18 

.25 

.09 

.16 

-.16 

.32 

34 

Active  Duty  Separation 

-.04 

.04 

-.05 

.02 

-.06 

.04 

Correlations  by  In-Service  and  Enlistment  Options 


7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

.25 

-.11 

.16 

.21 

.48 

.05 

.17 

.33 

.24 

.14 

.34 

.24 

-.09 

.19 

.20 

.30 

-.05 

.24 

.35 

.35 

.16 

.33 

.17 

-.21 

.09 

.10 

.28 

-.04 

.00 

.13 

.17 

-.04 

.09 

.24 

-.09 

-.09 

.35 

-.01 

-.27 

-.12 

.06 

.07 

-.07 

-.05 

-.21 

-.10 

-.09 

-.40 

-.07 

.09 

-.04 

-.21 

-.32 

-.14 

-.10 

.21 

-.20 

.10 

.26 

.35 

.02 

.02 

.27 

.27 

.02 

.17 

- 

-.01 

.23 

.15 

.32 

-.23 

.19 

.25 

.16 

.26 

.24 

.10 

- 

-.05 

.03 

-.02 

.01 

.18 

-.01 

.10 

.32 

.15 

.08 

-.04 

- 

.05 

.23 

-.06 

.20 

.11 

.16 

.15 

.13 

.22 

.00 

.01 

- 

.12 

.05 

.13 

.18 

.21 

.03 

.05 

.29 

-.02 

.16 

.16 

- 

.23 

.36 

.44 

.45 

.30 

.58 

-.21 

.02 

.01 

.02 

.31 

- 

.32 

.22 

.30 

.17 

.22 

.23 

.12 

.00 

.20 

.57 

.37 

- 

.33 

.33 

.49 

.50 

.40 

-.11 

.11 

.22 

.60 

.14 

.45 

- 

.54 

.23 

.44 

.18 

.04 

.09 

.23 

.59 

.30 

.44 

.52 

- 

.35 

.46 

.03 

.32 

-.06 

.11 

.24 

.42 

.40 

.24 

.35 

- 

.43 

.39 

.05 

-.01 

.19 

.64 

.15 

.54 

.59 

.51 

.33 

- 

.12 

.07 

.21 

-.10 

.06 

-.07 

-.12 

.08 

.05 

-.05 

.01 

-.01 

-.01 

-.04 

.05 

-.05 

.04 

.04 

-.02 

-.07 

-.01 

.04 

-.24 

-.16 

.09 

-.31 

-.28 

-.18 

-.40 

-.25 

-.15 

-.22 

-.34 

.00 

.03 

.02 

.03 

.02 

.04 

-.01 

-.02 

-.07 

-.03 

-.01 

.07 

.15 

-.01 

.08 

.07 

-.03 

.16 

.02 

.15 

.08 

.10 

-.05 

-.26 

.01 

.00 

-.02 

.09 

-.07 

-.11 

.03 

-.15 

-.09 

.02 

.03 

-.03 

-.15 

.23 

.02 

.15 

.17 

.02 

.14 

.28 

-.14 

-.11 

.14 

.05 

-.28 

-.08 

-.13 

-.21 

-.37 

-.15 

-.30 

.15 

-.10 

.04 

.20 

.08 

.08 

.03 

.00 

.04 

.07 

.08 

.11 

-.14 

.04 

.14 

.29 

-.01 

.14 

.25 

.08 

.15 

.21 

.13 

-.05 

.01 

.17 

.27 

.08 

.19 

.24 

.16 

.05 

.13 

.25 

-.06 

-.03 

.00 

.19 

-.01 

.19 

.06 

.16 

.12 

.25 

.45 

.09 

-.13 

.12 

.26 

-.01 

.20 

.26 

.31 

.14 

.36 

.30 

.06 

.02 

.18 

.33 

-.06 

.12 

.20 

.32 

.21 

.33 

-.12 

.00 

.04 

.03 

-.09 

-.01 

-.04 

.04 

.03 

.03 

.03 

Scale 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

1 

RBI:  Peer  Leadership 

.10 

.03 

.10 

.09 

-.16 

-.04 

.03 

-.02 

.15 

-.05 

.25 

-.01 

.19 

.08 

.19 

.12 

-.04 

2 

RBI:  Achievement 

.27 

-.06 

.09 

.09 

-.04 

-.11 

.08 

-.09 

.01 

-.02 

.13 

-.03 

.19 

.09 

.17 

.17 

.04 

3 

RBI:  Fitness  Motivation 

-.04 

.10 

.15 

.16 

-.01 

.09 

-.09 

-.18 

.21 

-.07 

.15 

.15 

.54 

.07 

.25 

.14 

-.05 

4 

RBI:  Stress  Tolerance 

.16 

-.02 

.07 

.04 

.00 

.06 

-.06 

-.13 

.14 

-.18 

.07 

.17 

.20 

.16 

.26 

.23 

.02 

5 

RBI:  Hostility  to  Authority 

-.23 

-.01 

.05 

.03 

.07 

-.05 

.07 

.10 

-.02 

.11 

.02 

-.05 

.05 

.01 

-.12 

-.14 

-.05 

6 

RBI:  Self-Efficacy 

.09 

-.02 

.05 

.03 

-.07 

-.04 

-.01 

-.23 

.10 

-.08 

.15 

.19 

.19 

.09 

.25 

.09 

.04 

7 

RBI:  Army  Affective  Commitment 

.04 

-.01 

.05 

.01 

-.11 

.03 

-.02 

-.12 

.03 

-.12 

-.03 

.28 

.11 

.02 

.29 

.19 

-.12 

8 

RBI:  Continuance  Commitment 

-.06 

-.08 

-.19 

-.18 

-.20 

-.08 

.11 

-.14 

-.12 

-.06 

-.17 

.06 

-.10 

-.08 

-.11 

.06 

.00 

9 

RBI:  Traditional  Values 

-.09 

-.02 

.06 

.09 

.10 

-.05 

.03 

.01 

.00 

.13 

-.01 

-.02 

.08 

-.22 

-.07 

-.10 

.03 

10 

RBI:  Validity  Scale 

.03 

-.05 

.03 

.01 

-.11 

-.03 

.02 

-.05 

.04 

-.11 

.10 

.03 

.13 

.05 

.06 

.18 

.03 

11 

Values  :  Rugged  Leadership 

.00 

-.14 

-.02 

.01 

-.24 

-.09 

.08 

-.14 

.00 

-.04 

.19 

.20 

.31 

.08 

.20 

.23 

-.09 

12 

Values  :  Flexibility/Choice 

-.11 

-.10 

-.10 

-.09 

-.17 

-.15 

.11 

-.09 

-.09 

.10 

.06 

-.05 

.11 

-.06 

-.18 

-.03 

.00 

13 

Values  :  Structure/Recognition 

.06 

-.01 

.06 

.08 

-.26 

.03 

-.04 

-.09 

.11 

-.16 

.17 

.04 

.05 

.09 

.06 

.03 

-.04 

14 

Values  :  Altruism/Benevolence 

.05 

-.06 

.04 

.05 

-.13 

-.04 

.02 

-.22 

.13 

.05 

.08 

-.10 

.14 

.16 

.08 

.08 

.05 

15 

Values  :  Skill  Development 

.07 

-.05 

-.09 

-.06 

-.09 

-.03 

.02 

-.13 

-.01 

.02 

.06 

.06 

.06 

.01 

.02 

.21 

.03 

16 

Values  :  Job  Security 

-.02 

-.07 

-.08 

-.07 

-.35 

-.07 

.08 

.01 

-.05 

-.01 

-.19 

.01 

-.04 

-.21 

.10 

.07 

.03 

17 

Values  :  Teamwork 

.06 

-.18 

.07 

.07 

-.32 

-.12 

.09 

-.16 

.08 

-.01 

.09 

.15 

.14 

.06 

.19 

.24 

.04 

18 

Situational  Judgment  Test 

- 

.13 

.08 

.16 

.05 

.07 

-.13 

.03 

-.01 

-.25 

.33 

.13 

-.04 

.19 

.33 

.20 

.03 

19 

Promotion  Rate  1st  Lieutenant 

.04 

- 

.11 

.13 

.09 

.85 

-.65 

.01 

-.14 

.14 

.68 

-.42 

.51 

-.09 

.01 

.07 

.36 

20 

Std.  Total  Awards 

- 

- 

- 

.92 

.01 

.12 

-.09 

-.20 

.14 

-.12 

.19 

.10 

.34 

.12 

.33 

.19 

-.27 

21 

Std.  Weighted  Awards 

- 

- 

- 

- 

.04 

.14 

-.09 

-.17 

.17 

-.10 

.24 

.07 

.36 

.12 

.29 

.14 

-.23 

22 

AFQT 

-.34 

.07 

- 

- 

- 

.10 

-.11 

-.01 

-.15 

.28 

-.02 

-.03 

-.10 

-.01 

-.21 

-.17 

.09 

23 

IOT  Graduation 

-.01 

.03 

- 

- 

-.02 

- 

-.80 

.02 

.00 

.16 

.77 

-.35 

.69 

-.02 

.05 

.06 

-.11 

24 

IOT  Recycles 

.10 

-.08 

- 

- 

-.02 

-.57 

- 

.15 

-.12 

.10 

-.64 

.14 

.55 

-.11 

-.04 

.00 

.11 

25 

Personal  Discipline 

-.09 

.19 

- 

- 

.12 

.00 

.07 

- 

.00 

.06 

.08 

-.16 

-.11 

.00 

-.24 

-.18 

-.12 

26 

Leadership 

.28 

.00 

- 

- 

.08 

.28 

-.24 

-.21 

- 

-.07 

.06 

.03 

.04 

.24 

.22 

.29 

.02 

27 

Career  Continuance  Intentions 

-.30 

.09 

- 

- 

.06 

.06 

-.04 

.05 

-.05 

- 

.06 

-.15 

.05 

-.24 

-.37 

-.48 

.47 

28 

Branch-Specific  Tech  Knowledge 

-.05 

-.13 

- 

- 

-.13 

-.11 

-.06 

.02 

.10 

-.10 

- 

.21 

.12 

.26 

.17 

.03 

-.12 

29 

Army-Wide  Tech  Knowledge 

-.04 

-.07 

- 

- 

-.07 

-.02 

-.13 

-.11 

.24 

-.10 

.50 

- 

.24 

.14 

.36 

.29 

.04 

30 

Physical  Fitness 

.11 

-.07 

- 

- 

-.07 

-.13 

.10 

-.06 

.13 

-.10 

.20 

.22 

- 

.04 

.32 

.30 

.17 

31 

Branch  Satisfaction 

-.04 

-.07 

- 

- 

-.07 

.07 

-.09 

-.12 

.04 

-.24 

.20 

.11 

-.05 

- 

.28 

.41 

-.19 

32 

Army  Commitment 

-.01 

-.15 

- 

- 

-.15 

-.01 

.16 

-.03 

-.01 

-.52 

.03 

.02 

.08 

.29 

- 

.62 

-.34 

33 

Morale 

.05 

-.11 

- 

- 

-.11 

-.01 

.12 

-.16 

-.01 

-.44 

.20 

.18 

.13 

.48 

.56 

- 

-.22 

34 

Active  Duty  Separation 

.03 

.35 

- 

- 

.09 

-.11 

.11 

-.12 

.03 

.46 

-.13 

.02 

.18 

-.20 

-.33 

-.23 

- 

Note.  Correlations  for  the  enlistment  option  sample  appear  above  the  diagonal;  correlations  for  the  in-service  sample  appear  below  the  diagonal,  n  =  368  for  enlistment;  n  =  390 
for  in-service.  IOT  =  Initial  Officer  Training,  AFQT  =  Armed  Forces  Qualification  Test.  Values  greater  than  .10  are  statistically  significant. 


